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Abstract 

This  thesis  is  an  application  of  decision  analysis  techniques  to  the  problem  of 
evaluating  competing  research  and  development  investments  and  selecting  the  best 
portfolio  from  among  the  alternatives.  In  contrast  to  the  current  ad  hoc  approach, 
decision  analysis  provides  an  explicit  and  easily  explained  rationale  for  investment 
choices,  complete  and  consistent  incorporation  of  multiple  objectives,  and  direct  insight 
into  the  effects  of  uncertain  investment  returns.  A  hierarchical  objective  decomposition 
was  used  to  capture  the  user's  preference  structure.  The  hierarchy  was  then  used  to 
develop  a  computer-based  decision  aid  to  allow  the  users  to  quickly  and  consistently 
evaluate  technology  investment  options.  The  decision  aid  includes  the  ability  to 
automatically  perform  several  deterministic  and  probabilistic  sensitivity  analyses. 
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APPLICATION  OF  DECISION  ANALYSIS  TO  EVALUATE  ALTERNATIVE 
RESEARCH  AND  DEVELOPMENT  INVESTMENTS 


1.  Introduction 


1.1  Background 

Today  engineers  must  justify  the  money  and  time  invested  in  research  and 
development  (R&D)  efforts.  They  must  also  focus  time  and  money  in  such  a  way  that  the 
efforts  provide  the  most  benefit.  One  reason  these  problems  occur  is  the  benefits  from 
R&D  projects  are  generally  not  immediately  realized.  Another  reason  it  is  difficult  to 
ascertain  the  benefits  is  that  there  are  multiple  payoffs  from  the  technologies.  These 
payoffs  may  affect  cost,  schedule,  and  performance.  To  identify  the  relational  effect 
between  the  attributes  of  schedule  and  cost,  we  can  graph  them.  However,  a  better 
method  to  understand  the  tradeoffs  of  cost,  schedule,  and  performance  is  through  the  use 
of  multiple  objective  decision  analysis.  Decision  analysis  articulates  the  value  of  the 
benefits  from  an  action  and  allows  the  engineer  to  explicitly  consider  the  aspects  that 
cause  one  R&D  investment  to  be  better  than  another. 
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1.2  Problem  Statement 


The  recently  created  Air  Force  Research  Laboratory  (AFRL)  is  a  combination  of 
all  the  United  States  Air  Force’s  R&D  capabilities.  The  mission  of  the  AFRL  is  “To  lead 
the  discovery,  development,  and  timely  transition  of  affordable,  integrated  technologies 
that  keep  our  Air  Force  the  best  in  the  world”  [AFRL,  1998a].  The  Air  Vehicles 
Directorate  is  one  of  ten  directorates  in  the  AFRL.  The  primary  focus  of  the  Air  Vehicles 
Directorate  is  to  improve  capabilities  in  current  Fixed  Wing  Air  Vehicles  and  also  develop 
revolutionary  Fixed  Wing  Air  Vehicle  Technologies  [AFRL,  1998b].  The  Integration 
Division  integrates  the  various  functional  areas  of  aircraft  performance,  in  the  Air  Vehicles 
Directorate  [AFRL  1998b]. 

The  Air  Vehicles  Integration  Division  is  developing  a  consolidated  program  for 
fixed-wing  vehicle  development  known  as  the  Conceptual  Aircraft  Systems  Design  and 
Analysis  Toolkit  or  CASDAT  [Carter,  1997].  CASDAT  will  be  able  to  conceptually  build 
a  notional  aircraft  from  an  initial  concept  and  then  produce  predicted  flight  data. 

CASDAT  will  also  be  able  to  predict  the  aircraft’s  mission  performance  characteristics  and 
also  show  variation  caused  by  future  technology  applied  to  the  notional  aircraft.  Thus, 
CASDAT  will  allow  an  engineer  to  visually  create  a  new  aircraft,  with  new  technologies, 
and  test  this  aircraft  against  both  aerodynamic  principles  and  mission  performance. 
Currently,  CASDAT  does  not  explicitly  address  the  issue  of  the  tradeoffs  between  cost, 
schedule,  and  performance.  This  research  will  address  this  shortcoming. 

An  analysis  tool  incorporating  multiple  objective  decision  analysis  will  allow  the 
laboratory  to  properly  address  cost,  schedule,  and  performance  tradeoffs,  as  well  as 
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provide  an  ability  to  defend  the  decisions.  Sensitivity  analysis  will  allow  the  laboratory  to 
investigate  the  importance  of  specific  technologies  as  well  as  the  strengths  and  weaknesses 
of  each.  The  analysis  tool  will  be  capable  of  handling  uncertainty,  since  the  trae  cost  or 
performance  of  a  technology  may  not  be  known. 


1.3  Thesis  Overview 

The  thesis  contains  three  themes  throughout  its  pages.  This  first  and  most 
important  is  that  this  is  the  documentation  of  the  research  provided  to  the  Air  Vehicles 
Integration  Division.  The  thesis  also  provides  a  basic  decision  analysis  overview  for  the 
Air  Vehicles  Integration  Division.  Finally  the  thesis  is  a  guide  to  use  and  understand  the 
analysis  tool  that  has  been  created.  These  three  themes  can  he  found  in  the  following 
chapters. 

Chapter  2  contains  a  general  discussion  of  the  principles  of  decision  analysis.  This 
chapter  begins  by  explaining  the  concepts  of  decision  analysis  that  are  important  to  this 
thesis,  then  moves  to  a  more  specific  discussion  of  the  previous  uses  of  decision  analysis 
to  help  solve  the  problem  of  R&D  investments. 

The  Air  Vehicles  Integration  Division’s  investment  problem  is  solved  by  the  model 
presented  in  chapter  3.  The  chapter  explains  why  the  model’s  hierarchy  is  structured  the 
way  it  is.  The  model  is  then  validated  by  explaining  how  the  important  principles  are 
valid.  Finally,  the  chapter  explains  the  model’s  implementation.  A  Microsoft  Excel 
program  that  enables  the  user  to  gain  insight  into  the  decision  analysis  process  is 
developed. 
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The  fourth  chapter  provides  an  example  that  verifies  the  model  and  illustrates  how 
the  model  can  be  used.  A  thorough  explanation  of  the  problem  leads  into  the 
interpretation  of  the  results.  These  results  include  a  deterministic  approach  and  a 
probabilistic  approach. 

Finally,  we  reach  conclusions  about  the  effectiveness  of  the  decision  analysis 
model.  This  fifth  and  final  chapter  also  explains  areas  to  be  improved  and  where  future 
research  can  help. 
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2.  Literature  Review 


This  chapter  focuses  on  the  important  literature  behind  this  thesis.  This  literature 
review  initially  focuses  on  the  broad  sources  of  information  that  develop  decision  analysis 
theory.  This  section  transitions  to  the  literature  that  directly  focuses  on  approaches  to  aid 
R&D  investment  decisions.  Appendk  A  includes  a  thorough  example  of  decision  analysis. 
This  chapter  discusses  important  areas  that  directly  affect  this  thesis. 


2.1  Pertinent  Literature 

2.1.1  General  Literature 

Before  discussing  the  literature  that  specifically  applies  to  this  thesis,  it  is  beneficial 
to  explain  value-focused  thinking  and  general  decision  analysis  topics.  Most  decision 
analysis  models  are  built  using  one  of  two  approaches  based  on  the  motivation  of  the 
analyst.  A  value-focused  model  lists  the  important  values  and  objectives.  The  second 
approach  is  known  as  alternative-focused  thinking  and  does  not  typically  identify  the 
important  critical  objectives  which  make  better  decisions  [Keeney,  1994:33].  This 
approach  lists  the  known  alternatives  and  then  compares  their  values.  In  other  words, 
only  the  good  and  bad  points  of  the  known  alternatives  are  identified  and  used  in  the 
model.  Alternative  focused  thinking  does  not  directly  support  creating  new  alternatives 
based  on  satisfying  important  objectives.  For  these  reasons  the  model  used  by  the  AFRL 
Air  Vehicles  Integration  Division  was  created  using  value  focused  thinking.  This  process 


5 


lists  the  important  objectives  and  then  expands  on  these  objectives  until  a  complete  model 
is  developed. 

Two  requirements  for  a  value  focused  thinking  hierarchy  are  that  the  individual 
building  blocks  be  collectively  exhaustive  and  mutually  exclusive  [Kirkwood,  1997: 17]. 
The  evaluation  measures  are  collectively  exhaustive  if  they  encompass  all  important 
attributes  of  the  decision,  and  therefore  model  all  of  the  areas  that  a  decision  maker  needs 
to  choose  one  technology  over  another.  For  the  hierarchy  to  be  mutually  exclusive,  none 
of  the  evaluation  measures  of  the  hierarchy  are  redundant  and  are  all  treated 
independently.  In  other  words,  the  evaluation  measures  do  not  double  count  important 
attributes,  and  they  are  all  separate  ideas.  These  two  properties  are  very  important 
because  they  insure  the  decision  is  being  properly  portrayed  and  also  that  the  weights  of 
the  evaluation  measures  are  correct  [Kirkwood,  1997: 17]. 

Kirkwood  also  writes  that  it  is  important  for  the  model  to  be  operable  and  of  small 
size.  Operability  means  that  the  user  understands  how  the  multiple  levels  of  the  hierarchy 
interact  [Kirkwood,  1997: 18].  This  is  very  important  for  implementation  because  if  the 
user  does  not  understand  the  weights  or  evaluation  measures  the  model  will  not  produce 
accurate  results.  Kirkwood  also  states  it  is  important  for  the  model  to  be  small.  This 
means  with  all  things  equal  in  the  model’s  representation,  the  smaller  hierarchy  is  better 
simply  because  it  is  easier  to  understand,  and  thus  involves  less  confusion. 

The  evaluation  measures  of  a  value  focused  thinking  model  must  also  have  mutual 
preferential  independence  to  use  an  additive  value  function  [Kirkwood,  1997:239].  This 
additive  value  function  allows  the  model  to  be  presented  as  a  hierarchy  with  multiple  tiers. 
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Also,  the  weight  for  each  evaluation  measure  is  the  multiplication  of  all  the  weights  from 
the  previous  tiers.  Mutual  preferential  independence  exists  when,  given  all  of  the 
evaluation  measures  are  partitioned  into  Y  and  Z  (where  Z  are  evaluation  measures  that 
have  common  levels,  and  Y  are  evaluation  measures  with  different  levels),  for  each  Y,  the 
rank  ordering  of  differing  alternatives  does  not  depend  on  the  evaluation  measures  in  Z 
[Kirkwood,  1997: 238].  In  other  words,  for  all  combinations  of  evaluation  measures,  the 
ordering  of  values  for  an  alternative  does  not  depend  on  evaluation  measures  that  are  the 
same. 

The  additive  value  function  can  be  used  when  mutual  preferential  independence 
has  been  established.  This  function  is, 

n 

Value  =  X  *^i 

i=l 

where  n  is  the  number  of  evaluation  measures,  w,  is  the  weight  of  the  i*  evaluation 
measure  and  Vi  is  the  value  of  the  i*  evaluation  measure. 

A  decision  analysis  contains  two  steps,  the  first  step  is  the  deterministic  stage 
which  assumes  the  data  is  correct  and  certain.  The  second  step  is  the  probabilistic  stage 
which  considers  uncertainty.  The  deterministic  stage  wiU  produce  results  that  can  be 
misleading  when  the  decision  alternatives  have  variability  in  their  evaluation  measures. 
Because  of  this  variability,  the  probabilistic  step  continues  the  analysis  by  including 
uncertainty  in  the  model.  The  expected  value,  variances,  and  risk  profiles  for  the 
alternatives  can  be  viewed,  once  the  uncertainty  is  included.  The  risk  profile  or 
cumulative  density  function  (CDF)  demonstrates  the  expected  value,  variance  of  the  data, 
and  can  display  dominance. 


7 


There  are  two  t5^es  of  dominance,  deterministic  and  stochastic.  Deterministic 
dominance  is  the  stronger  dominance  because  it  means  an  alternative  has  no  probability  of 
ever  having  a  value  less  than  another  [Clemen,  1996: 123].  Thus,  remove  the  dominated 
alternative  from  the  population  of  alternatives.  Stochastic  dominance  relies  more  on 
probabilities  [Clemen,  1996: 124].  It  means  that  the  value  of  an  alternative  for  each 
cumulative  probability  is  always  greater  than  another  alternative.  Therefore,  this 
alternative  always  has  a  probability  of  obtaining  more  value  than  the  dominated 
alternative.  This  is  weaker  dominance  because  there  is  often  a  chance  the  dominated 
alternative  can  have  more  value. 


2.1.2  Specific  Literature 

Rouse,  Boff,  and  Thomas  [1997:  389]  note  that,  the  difficulty  of  measuring  the 
value  from  R&D  investments  is  caused  by  the  wide  range  of  benefits  from  these 
investments.  To  measure  the  value  for  the  benefits  they  suggest  using  word  scales.  These 
word  scales  include  the  benefits  from  both  the  main  R&D  goals  and  also  the  byproducts 
that  the  R&D  investments  produce  [Rouse,  Boff,  and  Thomas,  1997:  393].  They  list 
seven  types  of  benefits  of  R&D  investments  which  range  from  very  tangible  to  less 
tangible  [Rouse,  Boff,  and  Thomas,  1997:  392]. 

•  Solution  not  otherwise  possible  (very  tangible) 

•  Acceptable  performance/  cost  not  otherwise  possible 

•  Performance/  cost  improvements 

•  Enhanced  customers/  user  perceptions  and  willingness  to  pay 

•  Cost  avoidance 

•  Mishap  avoidance 
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•  Increased  confidence  (less  tangible) 

This  research  will  focus  on  the  benefits  from  performance/  cost  improvements.  The  value 
from  the  byproducts  of  the  investments  will  not  be  specifically  addressed. 

The  uncertainty  analysis  or  probabilistic  phase  of  the  analysis  should  follow  the 
deterministic  phase  of  the  analysis.  This  allows  for  more  understanding  of  the  important 
characteristics  of  the  decision.  Barrager  and  Gildersleeve  discuss  how  to  incorporate 
uncertainty  of  R&D  costs  and  performance  data.  They  explain  the  uncertainty  in  the 
evaluation  measures  can  be  accounted  for  by  using  discrete  approximations  for  the 
differing  uncertainty.  They  also  suggest  only  using  three  to  six  evaluation  measures  in  the 
uncertainty  analysis  [Barrager  and  Gildersleeve,  1989: 180].  The  difference  in  the  number 
of  evaluation  measures  depends  on  the  degree  of  concern  for  the  particular  issues 
[Barrager  and  Gildersleeve,  1989: 181]. 

Clemen  recommends  the  use  of  the  Pearson-Tukey  approximation  to  discretely 
approximate  the  uncertainty  in  the  model.  He  states  this  technique  works  best  for 
approximating  symmetric  distributions  but  it  is  robust  enough  to  be  effective  for 
asymmetric  distributions  [Clemen,  1996: 278].  By  using  three  points  to  approximate  the 
continuous  environment  of  the  decision  space,  some  data  will  be  lost;  however,  the  results 
are  quite  accurate,  even  for  asymmetric  distributions.  The  Pearson-Tukey  approximation 
takes  the  5%,  50%  and  95%  fractiles  for  the  evaluation  measure  and  then  weights  this 
occurrence  of  having  a  0. 1 85, 0.63,  and  0.1 85  chance  of  happening  respectively.  A  5% 
fractile  is  the  value  such  that  the  cumulative  density  of  that  value  is  equal  to  0.05 


9 


(F(V)=0.05).  So  the  50%  fractile  is  the  point  where  50%  of  the  data  that  can  occur  is  less 
than  that  value. 

In  1996,  the  Air  Force  conducted  a  value  focused  thinking  study  which  focused  on 
what  technologies  the  Air  Force  needed  to  develop  to  dominate  air  and  space  in  the 
future.  This  study  was  documented  in  An  Operational  Analysis  for  2025  by  Jack  A. 
Jackson  Jr.,  Brian  L.  Jones,  and  Lee  J.  Lehmhuhl.  The  study  focused  on  comparing 
various  technologies  on  a  value  objective  hierarchy  against  several  alternate  futures.  The 
alternate  futures  used  the  same  value  hierarchy,  but  the  weights  were  adjusted  to  represent 
the  future  appropriately.  This  allowed  the  study  to  focus  on  what  technologies  have  the 
most  impact  over  the  multiple  predicted  futures.  This  research  will  use  methods  very 
similar  to  those  used  in  An  Operational  Analysis  for  2025. 

An  alternative  approach  to  the  multiple  attribute  value  function  model  of  R&D 
investment  decisions  is  the  analytic  hierarchy  process  (AHP).  AHP  was  not  used  because 
the  process  requires  comparing  the  performance  of  each  pair  of  competing  technologies 
for  each  evaluation  measure.  Since  the  client  is  able  to  quantify  estimates  of  performance 
and  understands  the  value  of  various  levels  of  the  evaluation  measures,  it  is  better  to  use 
the  multiple  attribute  value  function  [Belton,  1986: 13].  In  addition,  the  multiple  attribute 
value  function  is  able  to  evaluate  a  new  technology  by  merely  entering  the  new  scores 
instead  of  adjusting  all  the  comparisons  between  each  alternative  for  each  evaluation 
measure.  AHP  can  be  used  to  ascertain  the  weights  but  is  not  used  because  the  client  is 
comfortable  directly  scoring  the  weights  of  the  various  attributes.  So  a  multiple  attribute 
decision  analysis  model  has  been  created. 
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2.2  Conclusion 


The  insight  that  can  be  gained  by  properly  applying  multiple  objective  decision 
analysis  to  a  problem  is  invaluable.  To  properly  apply  decision  analysis,  the  model  must 
be  built  insuring  all  the  important  aspects  are  included,  and  not  double  counting  any  of 
these  aspects.  These  aspects  also  need  to  be  preferentially  independent  from  each  other, 
therefore,  they  do  not  cause  any  interaction  effect.  Many  decision  analysis  models  have 
been  created  over  the  years;  however,  there  has  been  a  lack  of  published  models  that  deal 
specifically  with  R&D  decisions.  Of  the  few  articles  that  have  been  published,  we  can  see 
that  the  difficulty  of  modeling  previous  decisions  has  been  to  describe  the  benefits  from 
the  investments.  The  literature  also  states  xmcertainty  in  the  decisions  can  be  handled  by 
standard  Pearson-Tukey  approximation.  The  next  step  in  the  thesis  is  to  actually  model 
the  decision  and  then  see  if  the  model  does  include  the  important  principles. 
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3.  Methodology 


This  chapter  focuses  on  how  the  model  is  designed  and  implemented.  The 
following  section  explains  the  design  of  the  model.  Then  the  assumptions  of  the  model  are 
checked  for  validity.  The  final  section  explains  the  implementation  of  the  model  in  an 
automated  computer  program  that  is  to  be  used  by  the  Air  Vehicles  Integration  Division. 

3.1  Model  Design 

The  first  concern  when  developing  a  multiple  objective  decision  analysis  tool  is  to 
correctly  model  the  decision.  The  Air  Vehicles  Directorate  has  been  directed  to  improve  6 
areas  of  aircraft  performance. 

•  Unit  production  cost 

•  Operation  and  Support  Cost 

•  Development  cost 

•  Air  Vehicle  Weight 

•  Lift/ Drag 

•  Controllable  Angle  of  Attack  envelope 

The  laboratory’s  technical  development  approach  (TDA)  lists  these  areas  [Carter,  1997]. 
The  TDA  is  given  to  the  laboratory  as  objectives  to  accomplish  in  the  development  of 
future  fixed  wing  vehicles  [Carter,  1997].  Initially,  these  six  improvement  goals  were  the 
basis  for  the  decision  hierarchy.  Thus,  we  have  the  following  initial  hierarchy  (Figure  3. 1); 
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Figure  3.1  Initial  hierarchy 


However,  this  problem  break-out  does  not  accurately  portray  the  decision  structure. 

Although  these  are  all  important  areas  of  the  decision,  they  are  not  collectively 
exhaustive.  In  other  words,  all  the  important  issues  of  the  decision  are  not  properly 
addressed.  First,  the  time  until  the  technology’s  production  is  not  included  in  the  model. 
Also,  the  time  the  laboratory  spends  developing  the  technology  is  not  addressed.  Since 
these  two  issues  are  capable  of  changing  the  decision,  they  must  be  included  in  the  model. 
Second,  all  the  cost  issues  of  the  decision  are  not  included.  In  spite  of  the  fact  that  costs 
associated  with  the  fixed  wing  aircraft  are  in  the  decision,  no  reference  to  the  laboratory’s 
cost  to  develop  the  technology  is  included.  Another  major  concern  of  the  TDA’s 
objectives  is  the  inability  to  truly  represent  all  the  important  aspects  of  aircraft 
performance  [Carter,  1997].  Some  of  the  technologies  developed  will  only  pertain  to 
stealth,  while  others  will  allow  for  more  thrust.  By  linnting  the  performance  issues  to  only 
lift,  drag,  weight,  and  angle  of  attack,  the  model  does  not  accurately  capture  the  rational 
for  choosing  one  technology  over  another.  A  final  concern  is  the  TDA  is  a  very  fluid 
document  which  is  changed  often.  By  including  additional  aspects  the  model  is  more 
adaptive  to  future  changes  in  the  TDA. 

The  objective  hierarchy  was  modified  after  extensive  consultations  with  the  Air 
Vehicles  Directorate.  Several  of  the  important  characteristics  that  would  result  in  a 
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preference  of  one  technology  over  another  were  highlighted  and  incorporated  in  the  final 
hierarchy  (Figure  3.2).  Appendix  B  defines  the  evaluation  measiu-es  that  are  used  in  figure 


3.2  and  throughout  this  thesis. 


Figure  3.2  Final  Hierarchy 


This  hierarchy  can  be  viewed  as  modeling  the  decision  from  two  perspectives.  The 
first  perspective  is  the  tradeoff  of  the  Air  Vehicles  Integration  Division’s  resources  against 
futvue  payoffs  of  aircraft  impacts.  The  second  perspective  contains  the  critical  aspects  of 
aircraft  impacts  which  are:  estimated  time  until  the  deployment  of  the  technology  (Initial 
Operating  Capability  or  IOC),  cost  to  implement  the  technology  on  a  fixed  wing  vehicle 
(research,  development,  testing,  and  evaluation  cost  to  the  laboratory  or  RDTE  Cost  lab), 
and  performance  changes  of  the  fixed  wing  aircraft  due  to  the  technology  (Aircraft 
Impacts).  The  second  perspective  is  focused  on  the  technology’s  impact  on  a  fixed  wing 
vehicle  with  a  specific  mission  profile,  whereas  the  first  perspective  encompasses  the 
resources  that  the  laboratory  has  to  spend  to  obtain  the  goals. 
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The  cost  to  implement  the  technology  is  divided  into  three  important  areas.  The 
first  area  is  the  cost  to  complete  the  technology’s  development  after  the  Air  Vehicles 
Integration  Division  releases  the  technology.  This  is  development  cost  on  the  hierarchy. 
The  second  area  is  the  cost  to  operate  and  support  (0«&S  cost)  a  fixed  wing  vehicle  with 
this  new  technology.  The  final  area  is  the  cost  to  purchase  a  fixed  wing  vehicle  with  this 
technology.  This  final  cost  is  referred  to  as  production  cost  on  the  hierarchy. 

The  performance  changes  are  also  divided  into  multiple  areas.  These  areas  include 
mass,  aero,  and  observability.  First  mass  involves  the  aspects  of  performance  which  are 
related  to  the  weight  of  the  aircraft.  The  empty  weight  of  the  aircraft  is  the  weight  of  the 
aircraft  with  no  fuel  or  munitions.  The  payload  weight  is  the  weight  that  the  aircraft 
carries  to  complete  the  mission.  The  more  bombs,  missiles,  and  ammunition  the  larger  this 
value.  Finally  the  take-off  gross  weight  (TOGW)  is  in  the  hierarchy  because  this  is  the 
weight  of  the  aircraft  when  it  is  fully  loaded  and  ready  to  take-off.  This  is  quite  different 
than  the  previous  two  measures. 

The  next  area  of  performance  is  aero.  Mainly,  the  aero  characteristics  involve  the 
lift,  drag,  thrust,  fuel  consumption,  and  maneuverability  of  the  aircraft.  Since  the  lift  of  the 
aircraft  changes  when  the  aircraft  has  external  carriage,  the  lift  has  multiple  evaluation 
measures.  This  is  also  trae  for  drag  because  it  is  important  to  know  the  drag  with  and 
without  external  carriage  and  also  when  the  aircraft  is  flying  at  subsonic  and  supersonic 
speeds.  The  fuel  consumption  is  a  standard  measure  taken  at  sea  level.  The 
maneuverability  is  measured  by  the  controllable  angle  of  attack,  which  was  an  original 
objective  from  the  TDA. 
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The  final  area  of  the  performance  section  of  the  hierarchy  is  observability,  which  is 
broken  into  results  from  radar  cross  section  (RCS)  tests  and  infrared  signature  (IR)  tests. 
Appendix  B  contains  more  detailed  definitions  of  the  evaluation  measures.  It  is  important 
to  notice  all  the  original  TDA  objectives  are  still  represented  in  the  updated  model; 
however  this  hierarchy  is  better  able  to  represent  the  total  effect  of  the  technology  on  the 
laboratory  and  also  on  the  Air  Force. 

3.2  Model  Validation 

The  validation  of  a  decision  analysis  problem  consists  of  two  steps.  The  first  step 
is  checking  that  the  model  correctly  portrays  the  decision.  The  second  step  of  validation  is 
checking  to  insure  all  the  assumptions  and  theoretical  basis  are  based  on  reality.  Several 
decision  makers  in  the  aeronautical  R&D  community  examined  the  hierarchy,  including  a 
representative  from  the  Aeronautical  Systems  Center  (ASC).  Since  ASC  is  the  Air  Force 
agency  which  continues  R&D  of  the  technologies  created  by  the  Air  Vehicles  Directorate 
it  deemed  necessary  to  get  their  participation.  Dr.  Squire  Brown,  an  Aeronautical 
Engineer  who  works  for  the  ASC  at  Wright  Patterson  Air  Force  Base,  Ohio,  verified  and 
validated  the  model.  Stating  that  it  accurately  represented  the  aspects  of  the  decision 
[Brown,  1997]. 
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3.2.1  Correct  Model 


A  decision  cannot  be  modeled  correcdy  if  the  model  does  include  the  proper  value 
structure.  This  decision  maker  also  needs  to  understand  the  issues  the  stake-holders  find 
important  Since  the  Air  Vehicles  Integration  Division  is  the  decision  maker,  they  need  to 
insure  they  are  using  the  proper  value  system.  Each  stake-holder  in  a  decision  may  have  a 
different  value  structure  so  each  stake-holder  needs  to  be  independendy  addressed.  Three 
sets  of  stake-holders  are  present  in  this  decision.  The  first  and  most  obvious  set  of  stake¬ 
holders  are  the  individuals  at  the  laboratory.  The  second  set  of  stake-holders  are  the 
future  pilots  that  will  benefit  from  the  new  technologies  that  the  laboratory  will  create. 
The  final  set  of  stake-holders  are  the  United  States  population  since  the  technology  will 
eventually  be  used  to  defend  the  country. 

If  the  future  pilot’s  value  structure  is  used,  it  would  include  evaluation  measures 
such  as  bombs  on  target,  lethality,  operational  readiness,  survivability,  vulnerability,  and 
other  measures  which  connect  the  technology  to  a  use  in  war.  If  the  value  structure 
represents  the  United  States  population,  the  only  concerns  will  be  the  effectiveness  to 
defend  the  country,  protect  its  assets,  and  the  cost  to  do  these. 

The  problem  of  whose  value  stracture  to  use,  has  been  answered  by  the 
laboratory’s  technical  development  approach  (TDA).  The  TDA  shows  potential  aircraft 
payoffs  if  the  initial  objectives  are  obtained.  The  aircraft  payoffs  include: 

•  Reduced  acquisition  costs 

•  Increased  lethality 

•  Increased  mission  range 
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•  Increased  operational  readiness 

•  Reduced  vulnerability 

These  are  the  values  of  the  future  pilot  and  are  directly  related  to  the  values  of  the  United 
States  population.  The  laboratory  does  not  directly  model  these  payoffs  because  some  are 
very  nebulous  and  all  are  dependent  upon  each  other.  By  incorporating  the  TDA’s  goals, 
the  laboratory  is  able  to  provide  important  payoffs  to  the  nation  and  also  justify  their  R&D 
investments.  Thus  the  laboratory’s  value  stracture  adequate  represents  all  of  the  stake¬ 
holders. 

The  value  of  human  life  is  a  concern  the  model  does  not  specifically  address.  This 
is  important  when  comparing  technologies  involving  an  unmanned  aircraft  versus  a  single 
pilot  aircraft.  The  unmanned  aircraft  does  not  directly  put  a  human  life  in  danger’s  way,  so 
there  is  less  cost  if  the  aircraft  is  lost.  Therefore,  the  decision  maker  must  include  this 
issue  into  the  model  to  make  the  final  decision.  This  issue  was  not  included  in  the  model 
because  it  is  controversial  to  place  a  value  or  dollar  amount  on  a  human  life. 


3.2.2  Validated  Model  Assumptions 

The  assumptions  of  the  model  are  truly  the  most  important  area  for  discussion 
because  if  an  assumption  is  incorrect,  the  results  will  be  misleading  and  invalid.  In  this 
subsection  the  principles  of  decision  analysis  are  validated  for  this  model. 

Since  this  model  has  several  ‘costs’  an  explanation  of  the  differences  of  each  ‘cost’ 
is  required.  The  Research,  Development,  Testing  and  Evaluation  (RDTE)  Cost  is  the  only 
true  expense  related  to  the  decision.  The  RDTE  cost  expresses  the  amount  of  money  that 
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the  Air  Vehicles  Directorate  will  spend  on  the  technology  to  develop  it  to  a  technical 
readiness  level  (TRL)  of  six.  A  TRL  equal  to  six  implies  that  the  technology  has  been 
demonstrated  in  a  model  or  prototype  environment.  The  TRL  levels  range  from  one  to 
nine  and  were  developed  by  the  National  Aeronautics  and  Space  Administration  (NASA) 
to  explain  the  stages  of  a  technology’s  development  [Carter,  1997].  The  other  costs  are 
project  costs  that  are  directly  tied  to  the  technology's  application  on  a  fixed  wing  vehicle. 
The  development  cost  is  the  cost  required  to  get  the  technology  from  a  TRL  of  six  to  a 
TRL  of  nine.  (A  TRL  equal  to  nine  implies  that  the  technology  is  actually  “flight  proven” 
through  successful  mission  operations.)  The  production  cost  and  operations  and  support 
cost  are  the  projected  costs  of  acquiring  and  using  the  fixed  wing  vehicle  with  the  new 
technology  onboard.  They  are  also  included  in  the  model  because  they  are  important 
benefits  of  the  proposed  technology. 

A  debated  issue  of  multiple  objective  analysis  models  is  whether  or  not  to  include 
cost  in  the  model  or  to  use  cost  for  a  cost  benefit  analysis  once  the  model  has  been 
created.  The  development,  operations  and  support,  and  production  cost  should  be 
included  in  the  model,  because  they  are  not  the  traditional  issues  facing  cost  and  are 
instead  a  result  of  the  technology’s  application.  Not  including  the  RDTE  cost  in  the 
model  is  similar  to  not  including  the  price  when  deciding  what  type  of  car  to  purchase  in 
the  future.  All  four  costs  are  independent  and  never  double  count  the  same  dollar.  For 
these  reasons  aU  four  cost  measures  are  included  in  the  model.  However,  in  chapter  four 
the  results  are  checked  to  see  if  they  are  the  same  using  a  cost  benefit  analysis  approach 
was  chosen. 
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A  concern  of  the  model  is  the  existence  of  attribute  independence  between  each  of 
the  evaluations  measures.  It  can  be  shown  that  some  of  the  evaluation  measures  are 
functional  related;  thus,  it  is  not  appropriate  to  include  some  of  the  technologies. 

However,  from  the  laboratory’s  perspective,  the  evaluation  measures  are  considered 
independent.  An  example  of  this  independence  is  the  fact  that  the  controllable  angle  of 
attack  can  be  increased  without  changing  the  lift  or  drag  coefficients.  Also,  the  take  off 
gross  weight  (TOGW)  can  be  increased  without  changing  the  empty  weight  of  the  aircraft. 
Some  technologies  will  affect  several  evaluation  measures,  but  the  independence  of  the 
evaluation  measures  is  not  an  issue. 

The  model  has  been  checked  to  insure  it  is  collectively  exhaustive  and  mutually 
exclusive.  This  has  been  done  by  asking  the  people  at  the  Air  Vehicles  Integration 
Division  if  two  technologies  were  identical  in  all  of  the  listed  evaluation  measures,  whether 
there  were  any  other  reasons  one  technology  is  preferred  over  another  technology.  This 
question  was  used  repeatedly  to  bring  the  model  to  its  present  state.  Once  there  were  no 
new  aspects  then  the  model  was  declared  collectively  exhaustive.  It  was  then  reviewed  to 
insure  that  no  attribute  was  double  counted  in  the  model.  This  insured  that  the  attributes 
were  mutually  exclusive.  Therefore,  the  model  is  both  collectively  exhaustive  and 
mutually  exclusive. 

The  model  also  includes  the  principles  of  operability  and  small  size.  The  model  is 
well  understood  by  the  decision  maker.  It  is  also  in  the  terms  and  linguistics  of  the 
decision  maker.  These  add  to  the  models  operability.  Also  the  model  attempts  to  use  the 
most  encompassing  attributes  to  insure  the  smallest  final  hierarchy  as  possible. 
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On  the  basis  of  thorough  discussions  with  the  Air  Vehicles  Integration  Division  it 
has  been  assumed  that  the  costs  are  all  mutually  preferentially  independent.  This 
assumption  has  been  largely  based  on  the  laboratory  view  that  all  the  evaluation  measures 
ate  independent  when  comparing  them  on  a  similar  airframe.  If  the  airframe  was  to 
significantly  change  this  assumption  will  no  longer  be  valid.  Therefore,  while  the 
assumption  is  valid  if  changing  the  avionics  on  two  different  F-1 17’s,  it  may  not  be  valid  if 
comparing  two  different  fixed  wing  aircraft,  such  as  comparing  the  F-1 6  to  the  F-15. 
Nevertheless,  aU  the  evaluation  measures  are  assumed  to  be  preferentially  independent  in 
this  model.  The  assumption  of  mutual  preference  independence  can  be  tested  by  using  a 
pair-wise  comparison  that  reduces  the  total  number  of  comparisons  made  [Kirkwood, 
1997:  239].  This  allows  for  the  use  of  an  additive  value  function  to  describe  the  model 
relations.  This  function  is  able  to  capture  and  weight  all  of  the  important  aspects  of  the 
R&D  investments  decision. 

If  the  technologies  are  compared  on  multiple  platforms  the  decisions  would  need 
to  be  separated.  However,  the  decision  maker  will  recognize  the  performance  of  the 
technology  when  applied  to  the  different  platforms.  This  insight  will  enable  the  decision 
maker  to  further  justify  the  technology  based  on  the  performance  across  multiple 
airframes. 


3.3  Model  Implementation 

An  automated  Microsoft  Excel  7.0  program  was  created  to  run  the  model.  This 
program  utilizes  the  visual  basic  programming  capability  of  Microsoft  Office  95.  Because 
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the  laboratory  is  constantly  defending  the  selection  of  various  technologies  the  automated 
program  is  user  friendly  to  help  provide  insight  to  the  relative  values  of  the  R&D  of 
technologies.  The  automated  program  has  five  main  components.  The  first  two 
components  allow  the  user  to  easily  adjust  the  value  functions  and  weights  for  each 
evaluation  measure.  The  last  three  components  provide  the  data  analysis  and  two  forms  of 
sensitivity  analysis. 

Specifically,  the  first  component  allows  the  user  to  adjust  the  value  function  for 
each  evaluation  measure.  The  value  function  is  assumed  to  be  an  exponential  value 
function,  as  described  in  Appendix  A.  Since  the  exponential  value  function  is  used  the 
model  is  limited  to  continuous  functions  with  linear  relations  or  smoothly  changing 
relations.  Also,  the  use  of  step  functions  for  the  value  function  are  not  allowed.  The  user 
provides  the  score  with  the  most  value  and  also  the  score  with  the  least  value  and  then 
adjusts  the  exponential  curves  by  entering  the  score  where  50%  of  the  value  is  obtained  or 
by  actually  entering  the  value  for  rho  (p).  The  program  provides  the  user  a  graph  of  the 
value  function  to  insure  that  the  value  has  been  correctly  modeled  for  each  evaluation 
measure.  If  this  graph  is  inconsistent  with  the  user's  value  they  are  easily  changed.  An 
example  screen  shot  of  the  program’s  evaluation  measure’s  component  is  shown  next 
(figure  3.3). 
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Figure  3.3  Screen  shot  of  evaluation  measure  component 


The  second  component  allows  the  user  to  adjust  the  weights  for  each  tier  of  the 
hierarchy  (figure  3.4).  The  user  is  shown  a  pie  chart  for  each  tier  and  then  prompted  if  it 
is  correct.  If  the  weights  are  incorrect  for  the  current  decision,  the  user  is  able  to  vary 
them  to  capture  the  proper  decision  structure.  This  component  also  has  ‘preset’ 
personalities.  These  are  predetermined  weights  that  represent  a  particular  mindset.  The 
personalities  include  a  person  that  weights  aircraft  performance  highly,  a  person  that 
considers  what  the  lab  is  spending  (both  time  and  money)  as  very  important,  and  a  person 
that  finds  the  total  cost  and  time  to  the  public  is  highly  important.  These  three 
personalities  cover  the  most  common  views  on  the  decision.  Another  feature  of  this 
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component  is  to  see  the  current  setting  of  weights.  This  feature  shows  the  total  break-out 
of  the  weight  for  each  evaluation  measure.  Also,  the  sum  of  all  20  evaluation  measures 
should  equal  one. 
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Figure  3.4  Screen  shot  of  weight  component 


The  third  component  provides  the  deterministic  data  analysis.  It  is  provided  in 
three  forms.  The  first  form  is  the  total  value  for  each  technology  shown  in  a  bar  chart. 

The  second  form  is  a  bar  chart  for  each  technology  color-coded  to  include  all  of  the 
evaluation  measures.  This  allows  the  user  to  see  where  the  value  for  each  technology 
comes  from.  The  final  form  condenses  the  bar  chart  to  only  show  the  value  from  the 
main  tiers.  So  only  eight  areas  are  shown  as  opposed  to  the  20  evaluation  measures.  This 


allows  the  user  to  easily  understand  the  important  issues  and  the  good/bad  areas  of  each 
technology  (figure  3.5). 


Figure  3.5  Screen  shot  of  data  evaluation  component 


The  fourth  component  varies  the  weights  for  each  evaluation  measure.  The  user  is 
prompted  to  select  which  evaluation  measure’s  weight  is  to  be  varied  from  0  %  to  100%. 
This  option  is  available  for  every  evaluation  measure.  The  result  is  shown  to  the  user  as  a 
line  graph  with  each  technology  listed  (figure  3.6).  The  graph  also  shows  what  the 
current  setting  is  for  the  weight.  Thus,  the  decision  maker  can  see  how  the  decision  will 
be  affected  by  variability  of  that  weight.  If  the  decision  is  insensitive  to  the  changes  then 
the  decision  needs  no  further  exploration  in  this  area.  If  the  decision  is  sensitive  to  the 


Figure  3.6  Screen  shot  of  sensitivity  analysis  component 


The  fifth  and  final  component  of  the  automated  analysis  program  incorporates 
uncertainty  analysis.  The  decision  maker  is  allowed  to  pick  up  to  five  evaluation 
measures  that  have  uncertainty.  The  user  needs  to  insure  that  the  low,  medium,  and  high 
data  points  for  each  uncertain  measure  are  entered  on  the  data  worksheet.  The  data  points 
correspond  to  the  5%,  50%  and  95%  fractiles  that  are  used  by  the  Pearson-Tukey 
approximation.  After  selecting  the  five  or  less  uncertain  measures,  the  computer  returns  a 
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CDF  for  the  user  to  interpret  (figure  3.7).  The  CDF  is  able  to  give  the  user  important 
information  about  mean,  variance,  deterministic  dominance,  and  stochastic  dominance. 
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Figure  3.7  Screen  shot  of  uncertainty  analysis  component 


The  limit  of  five  evaluation  measures  was  chosen  because  it  takes  a  Pentium  133 
computer  15  minutes  to  calculate  the  results  for  five  measures.  To  include  six  evaluation 
measures  the  computer  time  increases  to  45  minutes.  If  all  20  evaluation  measures  are 
included  in  the  uncertainty  analysis,  theoretically  it  would  take  over  400  years  to  finish 
calculating.  The  computer  requires  this  much  time  because  of  the  visual  displays  being 
utilized  by  the  program.  For  these  reasons  it  is  beneficial  to  limit  the  capability  of  the 
program.  If  more  than  five  uncertain  measures  are  checked  off  only  the  first  five  will  be 
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used  in  the  analysis.  However,  if  the  decision  maker  needs  to  include  more  uncertain 
evaluation  measures  the  problem  can  be  modeled  in  other  tools  specifically  tailored  for 
decision  analysis.  These  other  tools  can  use  Monte  Carlo  simulation  to  include  the 
uncertainty  with  minimal  time. 

The  program  has  one  other  feature  called  data  check.  This  simply  tells  the 
computer  how  many  technologies  are  being  compared.  This  is  used  to  correctly  scale  all 
of  the  graphs  that  are  used  and  also  to  limit  computer  compiling  time.  The  program  is 
currently  setup  to  compare  up  to  ten  technologies. 

All  six  components  are  accessed  from  the  initial  hierarchy  dialog  box.  This  dialog 
box  is  automatically  started  when  Microsoft  Excel  opens  the  file.  If  the  program  is 
stopped  while  the  file  is  still  open,  it  can  be  restarted  by  going  to  the  toolbar  and  selecting 
‘Tools’  and  then  clicking  on  ‘StartUp’.  This  shows  the  initial  hierarchy  dialog  box  and  all 
components  are  then  accessed  from  there.  Listings  of  the  program  and  functions  that  are 
called  by  the  program  are  located  in  Appendix  D. 

3.4  Conclusion 

To  conclude  this  chapter,  a  value  hierarchy  has  been  developed  to  solve  the 
investment  problem  facing  the  Air  Vehicle  Integration  Division.  This  model  incoiporates 
an  additive  value  function  using  a  simplified  computation  of  the  total  value  obtained  from 
each  alternative  based  on  important  evaluation  measures.  An  automated  decision 
modeling  program  has  been  created  to  help  the  laboratory  with  future  decisions.  This  tool 
is  applied  to  an  example  decision  in  the  next  chapter. 
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4.  Example 


The  fourth  chapter  is  focused  on  providing  and  analyzing  an  example  to  further 
check  the  validity  of  the  model.  The  analysis  tool  discussed  in  the  previous  chapter  will  be 
used  in  conjunction  with  CASDAT.  Specifically,  the  analysis  tool  will  provide  insight  to 
the  output  data  from  CASDAT.  However,  CASDAT  is  not  yet  fully  operational  so  this 
example  is  based  on  a  combination  of  real  and  fictional  data.  More  importandy,  this  is 
only  an  example  and  is  for  illustrative  purposes  only. 

The  chapter  is  divided  into  two  main  sections.  The  first  section  provides  the 
background  to  the  example.  It  explains  the  various  technologies  that  are  being  compared 
and  also  lists  the  important  data  for  each  technology.  The  second  section  is  the  analysis  of 
the  five  different  technologies  and  provides  important  conclusions  that  can  be  extracted 
from  the  data  through  the  use  of  the  decision  analysis  tool. 

4.1  Example  Background 

The  objective  of  this  thesis  is  to  create  a  model  which  accurately  includes  all  of  the 
important  (from  the  decision  maker's  viewpoint)  effects  of  a  technology  on  a  fixed  wing 
aircraft.  The  theoretical  basis  of  the  model  is  sound  but  that  does  not  imply  that  the  model 
will  produce  meaningful  results,  thus  an  example,  with  predictable  results,  has  been 
created  to  see  if  the  model  is  appropriate.  This  example  consists  of  creating  an 
uninhabited  strike  aircraft  as  a  baseline  and  then  applying  two  different  technologies  to  the 
aircraft.  Each  technology  has  two  different  levels  of  implementation.  This  creates  a  total 
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of  five  different  technologies  to  compare.  The  uninhabited  strike  aircraft  with  no 
technological  changes  will  be  referred  to  as  the  ‘baseline’  technology. 

The  uninhabited  strike  aircraft  was  designed  to  perform  a  suppression  of  enemy  air 
defenses  (SEAD)  mission.  This  is  a  mission  where  the  aircraft  is  used  to  attack  surface  to 
air  missiles  (SAMs)  and  other  targets  which  limit  our  ability  to  control  the  airspace  of  the 
battlefield.  The  strike  aircraft  will  be  similar  in  shape  to  the  B-2  bomber  but  will  be  a 
much  smaller  aircraft  (figure  4.1).  The  strike  aircraft  will  not  be  capable  of  supersonic 
cruise,  nor  will  it  cany  weapons  externally.  The  main  rationale  for  an  aircraft  of  this  type 
is  to  gain  an  early  dominance  of  the  airspace  over  the  battlefield  with  minimal  loss  of 
aircrews  and  resources. 

The  two  technologies  that  are  being  tested  on  the  uninhabited  strike  aircraft  are  an 
aeroelastic  wing  and  a  lower  structural  integrity  of  the  aircraft.  The  aeroelastic  wing 
involves  changing  the  shape  of  a  wing  in  order  to  improve  the  aerodynamic  capabihties  of 
the  aircraft  during  flight.  This  idea  is  based  on  the  early  aircraft  that  the  Wright  brothers 
flew  which  involved  wing-warping.  Thus,  a  wing  that  can  change  its  shape  during  flight 
would  be  able  to  produce  more  lift  during  take-off  then  reduce  the  parasitic  drag  caused 
by  a  highly  cambered  wing  during  the  cruise  portion  of  flight.  This  is  the  first  stage  of  the 
technology’s  development,  referred  to  as  ‘50%  aero’  because  the  technology  is  only  at  a 
50%  stage  of  maturity.  The  full  maturation  of  the  technology  involves  investing  more 
time  and  money,  and  results  in  an  aircraft  with  no  control  structures.  Instead  of  the 
normal  complement  of  flaps,  ailerons,  and  elevators,  the  aircraft  will  change  the  shape  of 
the  wing  to  change  direction  and  altitude.  This  is  referred  to  as  ‘  100%  areo’. 
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Figure  4. 1  Uninhabited  Strike  Aircraft 

The  technology  that  changes  the  structural  integrity  on  the  aircraft  results  in  the 
aircraft  having  a  lower  g-load  capability.  Normally  fighter  aircraft  are  designed  to 
withstand  approximately  9  Gs  of  stress.  By  reducing  this,  the  resulting  aircraft  will  be 
lighter,  and  thus  able  to  carry  heavier  payloads  and  more  fuel.  The  reduced  integrity  also 
results  in  a  cheaper  purchase  price  for  the  aircraft.  In  the  following  example  there  are  two 
levels  of  integrity  investigated.  The  first  level  is  reducing  the  stmctural  design  liiriit  to  6 
Gs.  This  level  has  already  been  implemented  on  the  F-1 17  stealth  fighter.  The  second 
level  is  reducing  the  design  g-load  to  3  Gs.  These  levels  are  considered  two  different 
technologies  and  are  referred  to  as  ‘6  Gs’  and  ‘3  Gs’  in  the  example. 

Most  of  the  data  was  supplied  by  the  Air  Vehicles  Integration  Division  using  a 
program  known  as  FLOPS  (Flight  Optimization  System)  which  is  a  component  of 
CASDAT.  FLOPS  is  able  to  provide  aeronautical  data,  and  mission  performance  data 
along  with  limited  cost  analysis  data.  The  data  that  FLOPS  could  not  provide  was 
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generated  through  discussions  with  the  client.  The  data  was  entered  into  the  model  and 
five  areas  of  concern  were  indicated  for  uncertainty  analysis.  The  data  used  in  the  model 
can  be  found  in  table  4.1. 


Table  4.1  Model  example  data 


Eval  Meas  \  Technology 

Baseline 

50%  Aero 

100%  Aero 

6  Gs 

3Gs 

Lab  RDTE  Cost 

25 

33 

35 

27 

34 

Time  of  program 

5 

6.5 

7 

5 

6.5 

IOC 

7 

7.5 

8 

7 

7 

Development  Cost 

35 

40 

43 

36 

38 

O&S  Cost 

2.5 

2.4 

2.4 

2.4 

2.3 

Production  Cost 

12.5 

12 

12 

11.5 

11 

RCS 

0.333 

0.25 

0.2 

0.333 

0.333 

IR 

2 

2 

2 

2 

2 

Empty  Weight 

12.4 

12.3 

12.1 

12 

11.7 

Payload  Weight 

6 

6 

6 

6 

6 

TOGW 

20 

21 

22 

19 

18 

Lift-Dirty 

32.6 

33.13 

33.13 

32.958 

32.786 

Lift-Clean 

32.6 

33.13 

33.13 

32.958 

32.786 

Drag-Subsonic-C  lean 

1.3 

1.4 

1.4 

1.2 

1.25 

Drag-Subsonic-Dirty 

1.3 

1.4 

1.4 

1.2 

1.25 

Drag-Supersonic-Clean 

N/A 

N/A 

N/A 

N/A 

N/A 

Drag-Supersonic-Dirty 

N/A 

N/A 

N/A 

N/A 

N/A 

Thrust 

18.73 

18.73 

18.73 

18.73 

18.73 

Fuel  Consumption 

6.73 

6.6 

6.58 

6.3 

6.05 

Maneuverability 

30 

33 

35 

29 

25 

The  lift  evaluation  measures  for  clean  and  dirty  are  the  same,  because  the  aircraft 
does  not  have  external  weapons  or  fuel  tanks.  Similar  to  lift,  drag  will  have  the  same 
results  for  clean  and  dirty  flight  In  addition,  the  aircraft  does  not  exceed  subsonic  flight, 
thus  there  are  no  results  for  the  supersonic  drag.  It  was  determined  that  the  RDTE  lab 
cost,  time  of  program,  IOC,  empty  weight  and  TOGW  were  all  uncertain  evaluation 
measures.  The  ffactile  values  for  the  uncertain  evaluation  measures  can  be  found  in  table 
4.2 
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Table  4.2  Uncertainty  data  for  model  example 


5%  fractile 

Baseline 

50Aero 

lOOAero 

6  Gs 

3Gs 

Lab  RDTE  Cost 

24 

28 

33 

25 

28 

Time  of  Program 

4.5 

3 

4.5 

3 

3.5 

IOC 

6.5 

6.5 

6.5 

6 

6.25 

Empty  Weight 

12.45 

12.35 

12.15 

12.05 

11.8 

TOGW 

21 

23 

23 

20.5 

20 

1  50%  fractile 

Lab  RDTE  Cost 

25 

33 

35 

27 

34 

Time  of  Program 

5 

6.5 

7 

5 

6.5 

IOC 

7 

7.5 

8 

7 

7 

Empty  Weight 

12.4 

12.3 

12.1 

12 

11.7 

TOGW 

20 

21 

22 

19 

18 

1  95%  fractile 

Lab  RDTE  Cost 

25.5 

34.5 

37 

32.5 

38 

Time  of  Program 

6 

8 

8.5 

8 

8.5 

IOC 

7.5 

9.5 

10 

9.5 

9.5 

Empty  Weight 

12.35 

12.25 

12.05 

11.7 

11.6 

TOGW 

19 

20 

21 

17.5 

16 

The  evaluation  measure  value  functions  are  assumed  to  be  exponential.  The 
following  value  function  is  the  Time  of  Program  evaluation  measure  (figure  4.2).  The 
figure  represents  diminishing  returns  to  the  laboratory,  because  the  p  equals  -3.  This  is 
only  one  of  twenty  evaluation  measures.  The  value  function  for  each  evaluation  measure 
is  listed  in  Appendix  C.  This  appendix  also  gives  the  high  and  low  level  for  each  value 
function  and  also  the  value  for  p. 
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Figure  4.2  Value  Function  of  Time  of  Program 

In  chapter  3  the  additive  value  function  for  this  example  was  shown.  This  function 
included  the  final  weight  settings  for  the  decision.  They  are  again  listed  in  table  4.3  along 
with  the  weights  of  the  evaluation  measures  at  each  tier.  This  was  all  provided  by  the 
client  and  used  in  the  following  example. 
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Table  4.3  Weights  used  in  the  example 


Tier 

Event 

IlilMill 

IBGil 

First 

Tier 

Aircraft  Impacts 

0.45 

RDTE  Cost 

0.35 

0.3500 

Time  of  Program 

0.20 

0.2000 

Aircraft  Impacts 
Tier 

Cost 

0.20 

IOC 

0.30 

0.1350 

Performance 

0.50 

Cost 

Tier 

Develop 

0.30 

0.0270 

O&S 

0.30 

0.0270 

Production 

0.40 

0.0360 

Performance 

Tier 

Mass 

0.20 

Aero 

0.40 

Observability 

0.40 

Mass 

Tier 

Empty  Weight 

0.40 

0.0180 

Payload 

0.40 

0.0180 

TOGW 

0.20 

0.0090 

Aero 

Tier 

Lift 

0.10 

Drag 

0.15 

Thrust 

0.15 

0.0135 

Fuel  Consumption 

0.30 

0.0270 

Maneuverability 

0.30 

0.0270 

Observability 

Tier 

RCS 

0.60 

0.0540 

IR 

0.40 

0.0360 

Lift 

Tier 

Dirty 

0.60 

0.0054 

Clean 

0.40 

0.0036 

Drag 

Tier 

Sub-Clean 

0.15 

0.0020 

0.30 

0.0041 

Super-Clean 

0.20 

0.0027 

Super-Dirty 

0.35 

0.0047 

Since  there  are  twenty  evaluation  measures,  the  additive  value  function  is  a  sum  of 
the  values  multiplied  by  the  weights  for  the  twenty  different  evaluation  measures.  These 


evaluation  measures  are  defined  in  appendix  B.  For  the  example  the  additive  value 


function  is, 


Value  =  0.35*VRDra+0.2*Vttae  +0.135*Vioc  +0.027*  Vdeveiop  +0.027*Vproduct 


+0.036*Vo&s  +0.054*Vrcs  +0.036*Vir  +0.018*Vew  +0.018*Vpayioad 
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+0.009*Vtogw  +0.0054*Vl(d)  +0.0036*Vl(c)  +0.002*VD(sub-c) 
+0.0041*VD(Sub-D)  +0.0027*VD(Super-C)  -f0.0047*VD(Super-D)+0.0135*Vd,™st 
+0.027*Vfiiel  +0.027*Vmaneuv. 

It  is  important  to  note  that  the  weights  of  some  evaluation  measures  dealing  with 
the  aerodynamics  of  the  air  vehicle  do  not  have  a  large  impact  on  the  model;  however,  the 
total  sum  of  the  weight  for  Aircraft  Impacts  is  0.45  and  the  total  sum  of  weights  for 
Performance  is  0.225.  Since,  the  decision  maker  does  not  weight  these  items  highly  in  the 
model,  the  lower  tiers  wiU  show  less  affect  on  the  model. 

4.2  Example 

Once  all  the  information  has  been  input  into  the  model,  the  model  can  begin  to 
provide  insight  to  the  decision  maker.  Initially,  the  decision  maker  cares  about  the 
technology  that  has  the  most  value,  and  where  that  value  comes  from.  The  bar  chart  in 
figure  4.3  shows  that  the  baseline  technology  has  the  most  value.  This  is  because  the 
aeroelastic  wing  and  structure  technologies  do  not  obtain  as  much  value  from  the  aircraft 
impacts  as  is  lost  from  RDTE  cost  and  time  of  program.  Thus,  the  best  decision  is  to  use 
the  baseline  case.  There  are  other  important  issues  that  can  be  interpreted  from  the  figure. 
First,  the  50%  aero  and  6  Gs  are  both  better  than  their  more  expensive  and  fully  matured 
counterparts.  Also,  the  6  Gs  technology  and  baseline  appear  to  be  close  in  value; 
however,  the  baseline  does  have  more  value.  Since  some  of  the  evaluation  measures  are 
uncertain,  the  decision  may  actually  change  under  different  but  possible  conditions. 
Finally,  it  appears  that  the  time  of  program  and  RDTE  lab  cost  are  the  driving  forces  to 
the  decision  maker. 
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Baseline  50%  Aero  100%  Aero  6Gs  3Gs 

Technologies 

Figure  4.3  Total  value  for  each  technology 


The  benefit  without  cost  is  found  by  not  including  the  laboratory  RDTE  cost  in  the 
total  value.  By  dividing  the  benefit  by  the  recendy  removed  RDTE  cost,  a  cost  benefit 
analysis  is  performed  (Figure  4.4).  This  figure  shows  the  decision  does  not  change  fi'om 
the  conclusion  drawn  from  figure  4.3;  therefore,  the  two  methods  are  equivalent.  Since 
they  produce  the  same  ordering  of  technologies,  including  the  cost  in  the  hierarchy  does 
not  invalidate  the  model. 
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Figure  4.4  Cost  Benefit  analysis  for  example 


The  next  step  of  the  analysis  procedure  would  be  to  investigate  the  sensitivity  of 
the  decision  to  changes  in  the  objective  weights.  Figure  4.3  shows  that  the  four  new 
technologies  have  better  aircraft  impacts  values  than  the  baseline.  Would  the  decision 
change  if  the  weight  of  the  aircraft  impacts  was  varied  from  0%  to  100%?  The  answer 
can  be  seen  in  figiure  4.5.  The  vertical  line  shows  the  current  weight  for  Aircraft  Impacts 
which  is  45%.  If  the  decision  maker  feels  the  weight  may  be  more  than  55%  then  there  is 
a  decision  change  to  the  6  Gs  technology.  Since  this  is  a  litde  change,  further 
investigation  may  provide  useful  insight.  This  is  important  because  it  appears  the  decision 
may  be  limited  to  selecting  the  baseline  or  6  Gs  technology.  There  is  another  change  at 
85%  but  this  is  unlikely  given  the  current  attitude  of  the  decision  maker.  It  is  also 
interesting  to  notice  that  the  100%  aero  technology  is  consistently  the  worst  technology. 
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This  is  not  yet  a  dominance  issue;  however,  it  does  indicate  that  the  technology  may  not 
be  the  best  solution. 


Figure  4.5  Impact  from  changing  the  weight  of  aircraft  impacts 

After  testing  the  sensitivity  of  the  decision,  by  varying  all  of  the  weights  of  the 
evaluation  measures,  it  was  concluded  that  the  only  weights  that  affected  the  decision 
were  time  of  program,  RDTE  cost,  and  aircraft  performance.  Changing  the  other 
evaluation  measure  weights  did  not  change  the  decision.  Figure  4.6  shows  how  the 
weight  for  performance  affects  the  decision  and  also  represents  how  changing  the  weight 
on  a  lower  tier  does  not  affect  the  decision.  The  result  from  the  varying  of  the  weight  of 
performance  is  typical  of  all  the  lower  tier  weights.  Thus,  the  decision  is  insensitive  to  the 
lower  tiers,  because  the  decision  does  not  change  when  the  weights  associated  with  the 
lower  tiers  change.  Therefore,  we  can  see  that  the  first  tier  is  the  only  tier  that  is  sensitive 
to  the  weights  from  the  decision  maker  in  this  example. 
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The  final  step  of  the  analysis  is  incorporating  uncertainty.  This  is  done  by  using 
the  Pearson-Tukey  approximation,  which  discretely  approximates  the  probability  using 
three  firactiles.  Since  there  are  5  different  uncertain  evaluation  measures  with  three  levels 
each  used  in  the  approximation,  there  are  a  total  of  243  discrete  events.  Figure  4.7  shows 
the  resulting  cumulative  probability  distribution  (CDF)  for  the  value  of  each  technology. 
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Figure  4.7  CDF  for  each  technology 

This  CDF  is  able  to  provide  several  important  characteristics  of  the  technologies. 
The  variance  in  the  technologies  is  found  by  calculating  the  slope  of  the  line,  where  a 
steeper  slope  means  less  variance.  Also,  upon  inspection  of  the  figure,  it  can  be  seen  that 
the  6  Gs  technology  is  able  to  get  more  value  than  any  other  technology;  however,  there 
is  only  a  25%  probability  of  this.  There  is  an  75%  probability  that  the  baseline  case  is  as 
good  or  better  than  the  6  Gs  technology.  It  is  also  apparent  the  aeroelastic  technologies 
and  the  3  Gs  technology  all  are  closer  in  their  variance  and  expected  value.  Table  4.4 
shows  the  expected  values  for  each  technology  which  is  found  from  the  CDF  and  is 
indicated  on  figure  4.7  by  the  color-coordinated  arrows. 

The  decision  between  the  baseline  and  6  Gs  technology  depends  on  the  risk 
tolerance  of  the  decision  maker.  This  is  because  the  baseline  technology  has  less  variance 
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in  its  value,  whereas  the  6  Gs  technology  is  able  to  do  very  well  but  it  can  also  do  poorly. 
Thus,  a  risk  seeking  individual  may  prefer  to  pick  the  6  Gs  technology  since  there  is  a 
chance  of  getting  more  value  for  the  investment.  On  the  other  hand,  a  more  conservative 
individual  may  choose  to  use  the  baseline  technology  since  its  value  has  less  variance. 

It  is  also  important  to  see  that  the  6  Gs  technology  has  more  value  than  the  other 
technologies  on  the  CDF.  This  is  because  the  6  Gs  technology  stochastically  dominates  all 
of  the  other  technologies  except  the  baseline  technology.  The  stochastic  dominance  is 
present  because  the  6  Gs  line  is  always  to  the  right  of  the  other  three  technologies.  The 
main  point  of  this  Stochastic  dominance  is  that  the  3  Gs  technology  is  not  going  to  be  a 
better  decision  than  the  6  Gs  technology.  Also,  the  6  Gs  technology  is  better  than  the 
aeroelastic  technologies  because  of  the  stochastic  dominance. 

Figure  4.7  also  shows  that  the  100%  aero  technology  can  be  removed  from  further 
analysis,  because  there  is  deterministic  dominance  between  the  baseline  technology  and  the 
100%  aero  technology.  This  dominance  means  the  value  for  the  baseline  case,  under  its 
worst  case  scenario,  is  still  better  than  the  100%  aero  technology  at  its  best  case  scenario. 
The  figure  illustrates  this  because  there  is  a  100%  probability  that  the  100%  aero 
technology  will  achieve  a  value  of  0.47  or  less,  and  the  baseline  technology  has  0% 
probability  of  obtaining  a  value  of  0.47  or  less.  This  is  important  to  the  decision  maker 
because  the  100%  aero  technology  is  never  a  better  decision  than  the  baseline  technology. 

Table  4.4  shows  the  expected  value  for  the  baseline  is  better  than  the  expected 
value  for  the  6  G’s.  The  expected  values,  from  table  4.4,  are  not  the  same  as  the  values 
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from  figure  4.3.  The  difference  in  the  results  is  because  table  4.4  includes  all  of  the 
probabilities  from  the  CDF. 


Table  4.4  Expected  value  from  the  uncertainty  analysis 


Technology 

E(V) 

Baseline 

0.530 

50%  Aero 

0.363 

100%  Aero 

0.329 

6G's 

0.500 

3G’s 

0.378 

4.3  Conclusion 

It  appears  the  best  decision  is  to  not  apply  the  technologies  to  the  ciurent  version 
of  the  uninhabited  strike  aircraft.  The  only  exemption  to  this  is  to  possibly  investigate 
further  into  the  reducing  the  structure  of  the  aircraft.  It  may  prove  to  be  more  beneficial 
to  design  the  structure  of  the  aircraft  at  some  g-level  between  the  current  9  Gs  and 
proposed  6  Gs.  More  importantly  the  aeroelastic  technologies  should  not  be  included  on 
this  airframe. 

When  these  results  were  presented  to  the  Air  Vehicles  Integration  Division,  many 
of  the  conclusions  were  verified  by  current  instincts  of  the  type  of  aircraft  that  the 
technologies  were  being  compared  on.  First,  the  aeroelastic  technologies  are  not  expected 
to  perform  well  on  an  aircraft  with  a  large  delta  wing.  It  is  also  expected  that  the  as  the 
aeroelastic  technology  is  applied  to  the  aircraft  the  value  of  the  technology  will  decrease. 
This  can  be  directly  seen  in  all  of  the  various  analysis  done  on  the  alternatives  in  this 
research.  Secondly,  since  this  aircraft  is  very  similar  to  the  F-117  it  is  not  surprising  that 
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the  ideal  structural  design  would  be  similar.  The  F- 1 17  is  currently  designed  to  withstand 
6  Gs.  This  iminhabited  Strike  aircraft  does  tend  to  show  more  value  from  having  less  than 
9  Gs  but  more  than  6  Gs.  Since  the  model  does  incorporate  the  common  perceptions  of 
how  the  vehicle  should  be  designed  it  is  verified  to  accurately  model  the  decision. 
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5.  Conclusions 


5.1  Summary 

The  problem  facing  the  Air  Vehicles  Integration  Division  centers  on  how  to  best 
allocate  resources  for  technology  development.  The  idea  is  to  fund  technologies  that 
provide  the  most  benefit  to  the  Air  Force.  The  laboratory  has  been  directed  to  accomplish 
various  objectives  listed  in  their  TDA;  however,  these  objectives  do  not  fuUy  represent  all 
of  the  capabilities  of  each  technology.  Therefore,  the  hierarchy  that  was  developed 
includes  the  original  objectives  but  it  also  includes  many  other  important  aspects  of  aircraft 
design. 

After  the  model  was  designed  it  needed  to  be  validated  and  implemented.  The 
model  was  checked  to  insure  all  of  the  important  principles  of  decision  analysis  were  not 
violated.  Then  this  validated  model  was  implemented  on  a  spreadsheet.  The  spreadsheet 
model  was  designed  to  be  very  user  friendly.  By  creating  an  automated  decision  tool  the 
decision  maker  is  able  to  test  various  technologies  with  minimal  effort.  Once  the  analysis 
has  been  performed  the  figures  can  be  transferred  to  other  applications  for  justification 
during  later  stages  of  the  decision  process. 

Once  the  model  was  implemented  on  a  spreadsheet  it  was  tested  for  accuracy.  By 
creating  a  realistic  example  the  model  was  tested  to  see  if  it  produces  logical  results.  The 
example  that  was  tested  had  predicable  results.  The  results  for  the  model  mirrored  the 
expected  results.  This  implies  that  the  model  does  accurately  portray  the  decision  making 
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issues  that  the  laboratory  faces.  Finally,  since  the  model  does  accurately  represent  the 
important  issues  that  face  the  decision  maker,  it  can  provide  useful  insight. 


5.2  Recommendations  and  Future  Research 

The  next  step  in  this  research  should  be  to  use  the  automated  decision  tool  on  a 
real  world  technology  decision.  The  decision  should  not  be  solely  based  on  the  results 
from  the  model,  but  the  model  should  provide  insight  to  the  decision  maker. 

The  most  important  area  for  future  research  is  to  check  the  assumption  of  mutual 
preferential  independence.  This  assumption  can  be  specifically  tested  by  using  pair-wise 
comparisons  between  each  combination  of  evaluation  measures.  This  is  difficult  as  well  as 
time  consuming  and  thus  not  performed  in  this  thesis;  however,  it  does  provide  a  solid 
defense  to  the  assumption.  Currently,  mutual  preference  independence  has  been  assumed 
based  on  the  laboratory’s  opinion  that  the  evaluation  measures  are  independent  so  long  as 
the  airframe  does  not  change.  Through  a  formal  investigation  in  this  matter  it  may  be 
concluded  that  this  assumption  is  to  narrow  and  that  the  technologies  can  be  compared  on 
different  airframes.  It  may  also  be  concluded  that  the  simple  airframe  assumption  is 
incorrect  and  more  restrictions  are  required  to  have  mutual  preference  independence. 

The  next  area  that  would  benefit  from  further  research  is  to  use  utility  instead  of 
value  for  the  evaluation  measures.  This  step  would  allow  the  risk  tolerance  of  the 
laboratory  to  be  input  to  the  model.  The  addition  of  risk  tolerance  simplifies  the 
uncertainty  analysis  by  incorporating  the  risk  personality  of  the  laboratory  in  the  model. 
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Another  area  that  requires  more  work  is  adding  the  value  of  human  life  into  the 
model.  This  is  a  very  debated  issue  since  a  dollar  amount  can  not  be  placed  on  a  human 
life  and  that  value  of  human  life  is  just  as  difficult.  The  issue  is  important  because  various 
technologies  will  require  one,  two,  or  even  three  personnel  on  the  aircraft.  As  the  number 
of  people  on  the  aircraft  increases,  the  loss  of  that  aircraft  becomes  more  devastating.  The 
most  difficult  aspect  to  capture  in  the  model  is  comparing  an  uninhabited  aircraft  to  a 
piloted  aircraft.  Currently  the  model  does  not  address  this  issue;  thus,  research  is  required 
to  be  able  to  tackle  this  shortcoming. 

The  model  could  improve  by  including  a  portfolio  optimization  routine.  This 
would  allow  the  laboratory,  though  the  selection  of  various  technologies,  to  obtain  the 
best  combination  of  attributes.  Certain  technologies  will  excel  in  different  areas.  Thus 
funding  the  technologies  in  such  a  way  that  different  aspects  are  exploited  a  better  use  of 
the  resources  is  obtained. 

The  final  area  that  could  use  improvement  would  be  to  include  the  value  that  is 
obtained  by  various  byproducts  that  the  research  creates.  Possible  byproducts  are  new 
technologies  applied  in  different  fields.  It  is  also  possible  to  spend  less  time  and  money  on 
similar  future  technologies.  To  conclude,  these  improvements  to  the  model  will  help 
provide  more  insight  to  the  decision  maker  but  if  they  are  not  included,  the  decision  maker 
needs  to  be  aware  of  these  additional  areas  of  concern. 
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Appendix  A 


Example  of  Multiple  Objective  Decision  Analysis 

The  following  example  is  adapted  from  Making  Hard  Decisions  by  Robert  T. 
Clemen  pages  535  to  552.  This  example  has  been  altered  to  better  illustrate  the  issues 
facing  the  Air  Force  Research  Laboratory,  Air  Vehicles  Integration  Division. 

Assume  we  need  to  buy  a  car  and  there  are  only  three  cars  available  for  the 
decision.  The  three  cars  are  known  as  Standard,  Norushi,  and  Portalo.  Initially  the 
decision  maker  states  the  most  important  factors  in  buying  a  car  are  cost  and  life  span. 
Cost  is  defined  as  purchase  price  and  life  span  is  defined  to  be  time  until  the  car  is  no 
longer  a  reliable  source  of  transportation.  Through  research  the  following  data  (table  A.l) 
was  collected  and  is  considered  certain; 


Table  A.  1  Initial  car  performance  data 


Model 

Cost  ($1000) 

Life  Span 
(years) 

Standard 

8 

6 

Norushi 

10 

9 

Portalo 

17 

12 

These  data  points  can  be  plotted  on  a  comparison  chart  (figure  A.l).  This  comparison 
chart  displays  the  region  which  is  most  ideal  since  for  less  money,  the  life  span  is  longer; 
however,  no  cars  are  in  this  region.  Thus,  a  selection  must  be  made  which  will  trade-off 
increased  cost  for  increased  life  span.  Multiple  objective  decision  analysis  will  be 
employed  to  understand  this  trade-off. 
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Figure  A.1  Comparison  of  car  data 

The  basic  organization  of  the  decision  can  be  broken  down  into  the  value  obtained  from 
the  cost  and  also  the  value  from  the  life  span.  Thus,  cost  and  life  span  are  considered  to 
be  evaluation  measures.  This  can  be  seen  in  figure  A.2. 


Figure  A.2  Initial  value  hierarchy 


For  each  evaluation  measure  there  is  an  equation  which  relates  the  measure  to  a  value  for 
this  decision.  The  evaluation  measures  are  seen  in  the  following  charts  (figures  A.3-4). 
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The  better  the  performance  measure,  the  closer  to  getting  ‘  1’  for  the  value.  The 
undesirable  levels  will  get  a  value  of  ‘O’.  It  is  common  to  use  an  exponential  function  to 
capture  the  value  function.  The  formula  to  do  this  for  an  increasing  function  (higher 
amounts  of  x  are  preferred)  is  (Kirkwood,  1997:  65), 


v(x)  = 


l-exp[-(x-Low)/p] 

1  -  exp[-(High  -  Low)  /  p 
X- Low 

- ;; -  ot 

High  —  Low 


otherwise 


If  the  function  is  decreasing,  in  other  words  lower  amounts  of  x  are  preferred  then  the 
formula  is  (Kirkwood,  1997:  66), 


vix)  = 


1  -  txp[-iHigh  -  x)  /  p] 

'  o 

1  -  &pp[-{High  -  Low)  /  p 

High  -X  ,  . 

- ; — —  Otherwise 

High  -  Low 


For  each  function  the  ‘High’  is  the  highest  level  of  x  that  is  important  to  the  chent  and 
‘Low’  is  the  lowest  level  of  x  which  is  important  to  the  client  (Kirkwood,  1997:66).  If  the 
p  is  equal  to  infinity  (or  negative  infinity)  then  the  value  function  is  exactly  linear  (figure 
A.3);  however  if  the  p  is  greater  than  ten  (or  less  than  negative  ten)  it  is  generally  linear. 
The  decreasing  payoff  as  seen  in  figure  A.4  is  from  a  p  that  is  about  3.  The  most  common 
way  to  obtain  the  value  for  p  is  to  find  midvalue  for  the  range.  This  is  the  level  that 
obtains  50%  of  the  value  to  the  client.  Using  the  midvalue  and  tables  provided  by 
Kirkwood,  a  p  is  able  to  be  found. 
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Figure  A.3  Cost  EM  function 


Evaluation  Measure 


Figure  A.4  Life  Span  EM  function 

By  finding  the  value  for  each  model  of  car  we  obtain  table  A.2,  which  illustrates  the 
importance  of  the  cost  and  life  span  to  the  decision  maker  and  allows  for  some 
understanding  of  the  tradeoffs. 
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Table  A.2  Value  from  Evaluation  Measures 


Model 

Cost 

C  Value 

■HM 

LS  Value 

Standard 

8 

1 

6 

0 

Norushi 

10 

0.778 

9 

0.656 

Portalo 

17 

0 

12 

0.897 

Next,  we  create  an  additive  value  function.  The  additive  value  function  ties  the 
two  evaluation  measures  together.  This  involves  obtaining  a  measure  of  the  relative 
importance  of  the  two  different  evaluation  measures.  For  this  example  their  importance  to 
the  decision  maker  is  equal;  therefore,  the  weight  associated  with  each  evaluation  measure 
equals  0.5  or  50%.  The  additive  value  function  for  this  example  is. 

Total  Value  —  Wcoat  ^  Vcost  WlJfe  span  *  V„fe  Span 

where  wi  is  the  weight  of  the  evaluation  measure  and  Vi  is  the  value  for  the  evaluation 
measure.  Since  the  weights  are  equal,  the  equation  reduces  to: 

Total  Value  =  0.5  *  Vcost  +  0.5  *  Vnfe  span- 

Because  we  have  the  values  for  each  car  in  table  A.2  we  are  able  to  ascertain  the  total 
value  for  each  car  (table  A.3).  The  best  choice  is  the  car  with  the  value  closest  to  ‘  1’.  In 
this  example,  the  Norushi  would  be  the  better  choice.  This  is  not  to  be  interpreted  as 
meaning  the  Norushi  is  50%  better  than  the  Portalo,  since  the  value  is  a  number  that  is 
50%  larger.  The  correct  interpretation  is  simply  the  Norushi  has  more  value  to  the 
decision  maker  than  the  Portalo. 
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Table  A.3  Total  value 


Model 

Value 

Standard 

0.50 

Norushi 

0.72 

Portalo 

0.45 

At  this  point,  the  decision  maker  may  decide  that  miles  per  gallon  (MPG)  also  has 
an  affect  on  the  decision.  By  adding  MPG  to  the  model  (figure  A.5)  we  now  have  three 
evaluation  measures  and  the  weight  must  now  be  divided  between  the  three.  Suppose  the 
decision  maker  finds  that  importance  of  MPG  is  worth  half  of  the  decision  criteria  and  the 
other  evaluation  measures  remain  of  equal  importance.  This  would  indicate  that  MPG 
receives  0.5  of  the  weight  and  cost  and  life  span  each  receive  0.25  for  their  weights.  The 
model  will  need  to  be  resolved  for  this  new  model  and  again  the  best  decision  will  be  the 
alternative  with  the  most  value. 


Figure  A.5  Total  value 


Sensitivity  analysis  from  a  decision  analysis  model  can  provide  a  good  deal  of 
insight  to  the  problem.  A  common  way  to  perform  sensitivity  analysis  is  to  vary  the 
weight  of  the  evaluation  measures  to  see  at  what  point  the  decision  is  changed.  This  can 
the  seen  in  figure  A.6.  Here  the  decision  is  changed  from  Standard  to  Norushi  if  the 
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weight  for  MPG  is  below  0.25.  The  decision  maker  is  faced  with  the  question  of  whether 
the  weight  for  MPG  (currently  at  0.5)  might  really  be  below  0.25.  If  this  is  not  a 
possibility,  then  Standard  is  a  good  choice.  If  the  weight  can  vary  to  below  0.25  then  the 
decision  maker  may  need  to  spend  more  time  on  clarifying  their  goals  and  determining 
their  importance.  This  assumption  specifically  addresses  if  the  assumptions  of  the  weights 
are  correct  and  if  not  when  it  matters. 


Another  important  area  of  sensitivity  analysis  is  incorporating  the  uncertain  data.  If 
the  decision  maker  can  state  the  probability  for  an  event  to  happen  or  understands  the 
standard  deviation  of  the  measure,  then  uncertainty  analysis  can  be  performed.  Suppose, 
for  example,  that  the  MPG  data  the  manufactures  state  are  not  completely  reliable.  Figure 
A.7  shows  an  example  of  a  cumulative  density  function  (CDF)  for  such  a  situation.  This 
figure  is  able  to  show  many  aspects  of  how  to  interpret  and  understand  the  importance  of 
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the  conclusions  that  uncertainty  analysis  can  provide.  First,  the  point  labeled  ‘A’  is  read 
as,  “there  is  a  40%  chance  that  the  Standard  car  will  have  a  value  of  0.32  or  less.”  Also, 
the  point  labeled  ‘B’  is  interpreted  as,  “there  is  a  90%  chance  that  Norushi  will  have  a 
value  of  0.49  or  less.”  A  important  conclusion  to  be  made  is  that  Standard  is  a  riskier 
alternative  than  both  Nomshi  and  Portalo.  This  is  because  Standard  can  have  less  value 
and  also  more  value  than  Norashi  and  Portalo.  From  merely  an  expected  value  point  of 

I 

view  the  Standard  is  a  better  decision;  however,  it  has  more  risk.  The  risk  tolerance  of  the 
individual  is  helpful  in  deciding  which  is  a  better  decision.  Is  the  individual  willing  to  pick 
an  alternative  with  more  risk  and  possibly  more  value,  than  an  alternative  with  less 


variation  in  its  value. 


This  example  has  no  dominance  of  any  type.  This  is  seen  because  the  standard  car 
can  have  more  value  than  any  other  car  but  it  can  also  have  less  value.  Standard  can  not  be 
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a  dominating  alternative  nor  can  it  be  a  dominated  alternative.  Similarly,  the  Portalo  can 
have  less  value  than  Norashi  but  it  can  also  have  more  value  than  Norushi.  Therefore, 
Portalo  is  not  dominated.  For  these  reasons  figure  A.7  does  not  contain  any  dominance 
information. 

The  reasons  to  use  multiple  objective  decision  analysis  in  this  manner  are  three 
fold.  First,  the  tool  of  decision  analysis  is  easy  to  understand  and  the  analyst  is  not  relying 
on  magic  black-box  analysis.  Second,  the  results  are  easy  to  justify  and  no  random 
numbers  are  being  used;  thus,  the  answer  is  repeatable  (i.e.  simulation  is  not  utilized  for  a 
one  time  decision).  Finally,  if  the  assumptions  are  challenged  the  model  can  be  adjusted  to 
incorporate  the  new  view  and  then  re-analyzed  to  see  if  the  final  decision  is  altered. 
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Appendix  B 


Definition  of  Evaluation  Measures 

RDTE  Cost  Lab:  Research,  Development,  Testing  and  Evaluation  Cost  to  the  laboratory. 
In  other  words,  the  total  cost  for  the  technology’s  development  spent  by  the  Air 
Vehicles  Directorate  in  current  year  dollars  to  get  the  technology  to  a  Technical 
Readiness  Level  (TRL)  of  six. 

Time  of  Program:  The  total  time,  in  years,  the  Air  Vehicles  Directorate  spends  to  get  a 
technology  to  a  TRL  of  six. 

Aircraft  Impacts:  The  impacts  of  the  new  technology  on  the  proposed  fixed  wing  vehicle. 
IOC:  Initial  Operating  Capability,  is  the  time,  in  years,  to  get  the  technology’s  TRL 
from  six  to  nine.  This  implies  the  time  from  when  the  technology  is  proven  in 
the  laboratory  until  the  time  that  that  technology  is  on  an  fixed  wing  vehicle 
with  an  operations  capability. 

Cost:  The  costs  of  the  aircraft  with  the  new  technology. 

Development  Cost:  The  cost  in  current  year  dollars  to  get  the  technology  from  a 
TRL  of  six  to  a  TRL  of  nine  on  the  chosen  fixed  wing  vehicle. 

Operation  and  Support  Cost:  The  annual  expected  cost  of  the  fixed  wing  vehicle  in 
squadron  service,  in  current  year  dollars. 

Production  Cost:  The  cost,  in  current  year  dollars,  to  acquire  the  fixed  wing 
aircraft  with  the  new  technology. 
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Performance:  This  encompasses  the  results  from  the  new  technology  applied  to  the 
fixed  wing  aircraft. 

Mass:  The  aspects  of  mass  of  the  fixed  wing  vehicle  with  the  new  technology 
Empty  Weight:  The  empty  weight  of  the  aircraft  with  the  new  technology.  The 
empty  weight  is  defined  as  the  weight  of  the  aircraft  with  no  fuel, 
armament,  or  crew;  however,  the  empty  fuel  tanks,  radar  systems,  and 
other  equipment  is  included. 

Payload  Weight:  The  weight  of  the  payload  of  the  fixed  wing  vehicle.  This 
includes  all  ammunition,  missiles,  and  bombs. 

TOGW:  Take  Off  Gross  Weight  of  the  fixed  wing  aircraft  with  the  new 
technology. 

Aerodynamics:  These  are  the  aspects  of  aerodynamics  of  the  fixed  wing  vehicle 
with  the  new  technology. 

Lift:  This  combines  all  the  important  lift  characteristics. 

Lift  (dirty):  The  ingress  cruise  lift.  All  weapons  and  drop  tanks  are 

attached.  An  appropriate  altitude  is  picked  for  all  technologies. 

Lift  (clean):  The  lift  generated  of  the  aircraft  with  no  weapons,  pylons,  or 
drop  tanks.  This  is  ‘cleaner’  than  the  egress  portion  of  the  mission. 
Drag:  This  is  the  drag  for  the  fixed  wing  aircraft. 

Drag  Subsonic  (clean):  Drag  from  the  aircraft  at  a  subsonic  cruise  with  no 
external  carriage.  This  is  taken  at  the  same  time  as  lift  (clean). 
Drag  Subsonic  (dirty):  Drag  from  the  aircraft  at  a  subsonic  cruise  with  all 
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external  carriage. 

Drag  Supersonic  (clean):  Drag  from  the  aircraft  at  a  supersonic  cruise  with 
no  external  carriage. 

Drag  Supersonic  (dirty):  Drag  from  the  aircraft  at  a  subsonic  craise  with  all 
external  carriage. 

Thrust:  The  fixed  wing  aircraft  thrust  at  sea-level-static. 

Fuel  Consumption:  The  fuel  consumption  at  sea-level-static. 

Maneuverability:  Controllable  angle  of  attack 
Observability:  The  aircraft’s  ability  to  not  be  detected. 

Radar  Cross  Section:  The  frontal  radar  cross  section  of  the  fixed  wing  aircraft. 
Infrared  Signature:  The  infrared  signature  of  the  fixed  wing  aircraft. 
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Appendix  C 


Evaluation  Measure  Functions 


iLab  RDTE  Cost  1 

X 

Used 

Low: 

20 

High: 

40 

Mono: 

Decreasing 

Rho: 

Infinity 

Time  of  Program  1 

X 

Used 

Low: 

3 

High: 

9 

Mono: 

Decreasing 

Rho: 

-3 

iOC 

X 

Used 

Low: 

6 

High: 

10 

Mono: 

Decreasing 

Rho: 

-1.64 

iDeveiopment  Cost 

X 

Used 

Low: 

30 

High: 

50 

Mono: 

Decreasing 

Rho: 

8.2 

60 


O&S  Cost 

X 

Used 

Low: 

2 

High: 

2.6 

Mono: 

Decreasing 

Rho: 

Infinity 

1  Production  Cost 

X 

Used 

Low: 

10 

High: 

13 

Mono: 

Decreasing 

Rho: 

-0.756 

iRadar  Cross  Section  | 

X 

Used 

Low: 

0 

High: 

2 

Mono: 

Decreasing 

Rho: 

-0.378 

lIR  Signature  I 

X 

Used 

Low: 

1 

High: 

5 

Mono: 

Decreasing 

Rho: 

*1.64 

Square  Meters 
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Lift  Clean 

X 

Used 

Low: 

32.5 

High: 

33.5 

Mono: 

Increasing 

Rho: 

-0.324 

io<Oh-ooo)t-c\|cq'^ 

cvicvicvicvicvicocococo 

cococococococococo 


Pounds  (thousands) 


Pounds  (thousands) 


iDrag  Subsonic  Dirty  I 

X 

Used 

Low: 

1 

High: 

1.5 

Mono: 

Decreasing 

Rho: 

Infinity 

IDrag  Supersonic  Clean  I 

X 

Used 

Low: 

1 

High: 

1.5 

Mono: 

Decreasing 

Rho: 

Infinity 
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1.50 -K  1-50-Jt  1.50  33.5  J 


Pounds  (thousands) 


Thrust 

X 

Used 

Low: 

15 

High: 

20 

Mono; 

Increasing 

Rho: 

infinity 

I  Fuel  Consumption  | 

X 

Used 

Low; 

6 

High: 

6.8 

Mono; 

Decreasing 

Rho: 

-0.328 

iManeuverability  I 

X 

Used 

Low: 

20 

High: 

40 

Mono: 

Increasing 

Rho: 

-8.2 

0<Oi-h-CVJCOCOO)’^p 

idid<D<dr<t^cda)0)cj 
'P-T-'I-'T-'T-'l—  'T-'f-l-CVJ 


Pounds  (thousands) 


o 

a> 

GO 

h- 

<o 

CO 

CM 

t- 

o 

o 

q 

CM 

cq 

lO 

<q 

CO 

cd 

(d 

<d 

<d 

<d 

cd 

cd 

cd 

cd 

cd 

Pounds  (thousands) 


AOA 
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Appendix  D 


Visual  Basic  Program  for  Automated  Decision  Tool 

Program 

The  following  program  is  the  backbone  of  the  automated  analysis  program  that 
was  developed  for  the  AFRL  Air  Vehicles  Integration  Division,  The  code  runs  as  a 
module  in  Microsoft  Office  95  Excel  7.0.  The  code  documentation  explain  what  each 
subroutine  is  doing  and  also  important  variables.  Visual  Basic  does  not  require  declaring 
all  variables  so  only  the  important  ones  have  been  chosen  in  this  program. 


’  These  are  variables  that  are  used  throughout  the  program 
'  Index  keeps  track  of  the  rows  when  required 
Public  Index  As  Integer 

’  Counter  is  a  standard  count  for  loops 
PubUc  Counter  As  Integer 

'  Num  is  used  to  specify  the  number  of  technologies  being  compared 
Public  Num  As  Integer 

’  This  causes  the  program  to  automatically  execute  when  the  file  is  opened 
'  Basically  it  calls  the  startup  dialog  box  and  then  lets  the  dialog 
’  box  do  die  work  from  there. 

'  When  exited  the  user  is  told  how  to  restart  the  program 

Sub  Auto_C)penO 
Startup 

MsgBox  ("To  return  to  the  program,  go  to  Tools  then  StartUp") 

End  Sub 

’  This  allows  the  user  to  call  the  program  after  the  excel  worksheet 
'  has  been  activated 
Sub  StartUpO 

DialogSheets("Startup").Show 
End  Sub 
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’  EvalStart_Click  Macro 

’  If  the  evaluation  measure  button  is  selected  from  die 
'  startup  dialog  box  then  the  following  code  is  executed. 

Sub  EvalStart_QickO 
EvalMeasures 
End  Sub 

’  WeightsStart_Click  Macro 
'  Similar  to  the  evaluation  measures,  however  this  calls 
'  a  dialog  box  to  help  the  user  select  how 
’  to  set  the  weights 

Sub  WeightsStart_Click() 

DialogSheets(’'Weightpersonalities").Show 
End  Sub 

'  this  is  the  data  check  code. 

’  it  merely  finds  the  number  of  technologies  which 
’  will  be  evaluated  and  saves  the  number  in  a  cell. 

'  The  program  is  limited  to  evaluating  10  technologies. 

Sub  DataEntryStart_ClickO 
Num  =  AppUcation-InputBox  _ 

("The  number  of  technologies  to  be  compared.", ,  "5",  Type:=l) 
Do  While  Num  >  10 

Num  =  Application.InputBox  _ 

("The  number  of  technologies  must  be  less  than  or  equal  to  10." 
,,"5",Type:=l) 

Loop 

Range("Model!$A$l").Value  =  Num 
End  Sub 

I 

’  DataEvalStart_Click  Macro 
'  This  calls  a  dialog  box  to  show  the  Data  evaluation 
'  when  the  button  is  selected 

Sub  DataEvalStart_QickO 
DialogSheets("dialogeval").Show 
End  Sub 

t 

'  SenAnalStart_Click  Macro 
'  This  pulls  up  the  box  to  choose  the  sensitivity 
'  analysis  dialog  box 

Sub  SenAnalStart_ClickO 
DialogSheets("SenAnal").Show 
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End  Sub 


'  UncerStart_Click  Macro 

’  This  is  the  start-up  code  if  the  uncertainty  analysis  button  is 
’  selected 

Sub  UncerStart_Click() 

DialogSheets("Uncertainty").Show 
End  Sub 

'the  execution  and  ordering  for  the  eval  measures  code 
'each  evaluation  measure  must  have  it’s  own  dialog  box 
’also  the  evaluation  measures  are  equally  spaced  at  12 
'rows  a  part  on  the  EvaluationMeasures  worksheet 

Sub  EvalMeasuresO 
Index  =1 

I>ialogSheets('’LabRDTE").Show 
Index  =  Index  +  12 
I>ialogSheets("timeofprogram'’).Show 
Index  =  Index  + 12 
DialogSheets("IOC").Show 
Index  =  Index  +  12 
DialogSheets('’Develop").Show 
Index  =  Index  +  12 
DialogSheets('’0&S").Show 
Index  =  Index  +  12 
DialogSheets("Product").Show 
Index  =  Index  +  12 
DialogSheets("RCS").Show 
Index  =  Index  +  12 
DialogSheets('’IR").Show 
Index  =  Index  +  12 
DialogSheets("EW").Show 
Index  =  Index  +12 
DialogSheets('’Payload").Show 
Index  =  Index  +  12 
DialogSheets('’TOGW").Show 
Index  =  Index  +  12 
DialogSheetsC ’Dirty  ").Show 
Index  =  Index  +  12 
DialogSheets("Qean").Show 
Index  =  Index  +  12 
DialogSheets("Sub").Show 
Index  =  Index  +  12 
DialogSheets("Sub-D").Show 
Index  =  Index  +  12 
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DialogSheets("Super").Show 
Index  =  Index  +  12 
DiaIogSheets("Super-D").Show 
Index  =  Index  +  12 
DialogSheets("Thrust").Show 
Index  =  Index  + 12 
DialogSheets("Fuel").Show 
Index  =  Index  +  12 
DialogSheets("Maneuv").Show 
Index  =  Index  +  12 
End  Sub 


’  EvalMeas_Show  Macro 

'  The  initialization  of  the  evaluation  measures  infonnation  box 
'  This  insures  that  the  conect  infonnation  is  requested  and  input 
’  for  each  evaluation  measure.  It  also  puts  in  the  correct  default 
'  values 

Sub  EvalMeas_Show() 

Sheets("EvaluatioiiMeasures’’).Activate 
With  ActiveDialog 
.EditBoxes.Text  =  "" 

.OptioriButtons("ihoButton").Value  =  xlOn 
.EditBoxes("novalue").InputType  =  xINumber 
.EditBoxes("allvalue").InputType  =  xINumber 
'  since  the  code  fimction  (ValueE)  needs  the  information  of 
'  increasing/decreasing  tins  must  be  used  in  the  code 

If  UCase(CeIls(Index  +  4, 3))  =  "INCREASING’’  Then 
.EditBoxes(’’novalue'’).Text  =  Cells(lndex  +  2, 4) 
.EditBoxes(’'allvalue’’).Text  =  Cells(Index  +»3, 4) 

Else 

.EditBoxes('’novalue'’).Text  =  Cells(Index  +  3,4) 
.EditBoxes(’’allvaIue’’).Text  =  Cells(lndex  +  2, 4) 

Endlf 

.EditBoxes("rhovaIue’’).Text  =  Cells(lndex  +  5, 4) 

End  With 
End  Sub 

’  OKbutton_Click  Macro 

’  once  the  data  is  input  the  information  must  be  stored  in  a  new  location 
'  it  is  also  important  to  insure  the  increasing/decreasing  is  handled 
’  properly 

Sub  OKbutton_Click() 

Sheets(’’EvaluatioriMeasures’').Activate 

lower  =  CDbl(DialogSheets(’’EvalDialog’’).EditBoxes(’’novalue’’).Text) 
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higher  =  CDbl(DialogSheets("EvalDialog").EditBoxes("allvalue").Text) 

If  DialogSheets("EvaIDialog").OplionButtons("ihoButton").Value  =  xlOn  Then 
rtiotemp  =  DialogSheets("EvalDialog").EditBoxes("ihovalue").Text 
Else 

R  =  Cower  -  DialogSheets("EvalDialog").EditBoxes("midvalue").  _ 

Text)  /  Cower  -  higher) 

Cells(Index  +  6, 3).Select 
ActiveCell.FonnulaRlCl  =  R 
Worksheets("EvaluationMeasures").Calculate 
CellsCndex  +  6, 4),Select 
Rho  =  ActiveCeU.Value 
If  Rho  =  "Infinity"  Then 
ihotemp  =  "Ii^nity" 

Else 

ihotemp  =  CDbl(Rho)  *  AbsCower  -  higher) 

Endlf 
End  If 

If  higher  <  lower  Then 
temp  =  higher 
higher  =  lower 
lower  =  temp 
mono  =  "Decreasing" 

Else 

mono  =  "Increasing" 

End  If 

’  this  is  the  actual  data  input  to  the  worksheet 
CeUsCndex  +  2, 4). Select 
ActiveCeU-FormulaRlCl  =  lower 
CellsCndex  +  3, 4).Select 
ActiveCell.FormulaRlCl  =  higher 
CeUsCndex  +  4, 4).Select 
ActiveCeU.FormulaRlCl  =  mono 
CeUsCndex  +  5, 4).Select 
ActiveCeUPormulaRlCl  =  rhotemp 
Woiksheets("EvaluationMeasures").Calculate 
End  Sub 


’  WeightDialog_OKButton_CUck  Macro 
'  SimUar  for  the  evaluation  measures,  this  takes  the  new  data  and 
’  stores  it  on  the  worksheet.  It  also  checks  for  mistakes  in  the  input. 

Sub  WeightEtialog_OKButton_ClickO 
Sheets("Weights").Activate 

tempi  =  DialogSheets("WeightDialog").EditBoxes("textr').Text 
CeUs(Counter,  2).Select 
If  ActiveCeU.FonnulaRlCl  =  "N/A"  Ihen 
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tempi  =  "0" 

End  If 

temp2  =  DialogSheets("WeightDialog")EditBoxes("text2").Text 
Cells(Counter  +  1, 2).Select 
If  ActiveCeU.FormulaRlCl  =  "N/A"  Hien 
temp2  =  "0" 

End  If 

temp3  =  DialogSheets("WeightI>ialog").EditBoxes("text3").Text 
CeUs(Counter  +  2, 2).SeIect 
If  ActiveCelI.FomndaRlCl  =  "N/A"  Then 
temp3  =  "0" 

End  If 

temp4  =  DiaIogSheets("WeightDialog").EditBoxes("text4").Text 
CeUs(Counter  +  3, 2).Select 
If  ActiveCeUPoimulaRlCl  =  "N/A"  Then 
temp4="0" 

End  If 

temp5  =  E)ialogSheets("WeightDialog").EditBoxes("text5").Text 
Cells(Counter  +  4, 2).SeIect 
If  ActiveCellPoraiulaRlCl  =  "N/A"  Then 
temp5  =  "0" 

End  If 

Cells(Counter,  4).Select 
ActiveCell.FormuIaRlCl  =  tempi 
Cells(Counter  +  1, 4).Select 
ActiveCeU-FonnulaRlCl  =  temp2 
Cells(Counter  +  2, 4).Select 
ActiveCell.FonnulaRlCl  =  temp3 
Cells(Coimter  +  3, 4).Select 
ActiveCell.FonnulaRlCl  =  temp4 
Cells(Counter  +  4, 4).Select 
ActiveCeU.FormulaRlCl  =  temp5 
Worksheets("Weights").Calculate 
End  Sub 


’  Control  the  execution  of  weights  assessment 
’  if  own  weight  adjustment  is  selected  then  the  weights  subroutine 
’  is  called.  If  the  performance  based  assessment  or  cost  assessments 
'  are  selected  then  the  data  from  weight-personality  is  used 

'  own_Click  Macro 

'  this  allows  the  individual  to  select  the  weights 

Sub  own_Click:0 
Weights 
End  Sub 
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’  current_Click  Macro 

’  This  allows  the  user  to  see  the  current  settings  for  each  weight. 
'  This  is  the  cumulative  break-down  of  the  weight 

Sub  current_Click() 

Calculate 

DialogSheets("weightsetting").Show 
End  Sub 


'  weightsetting_DialogFramel_Show  Macro 

’  This  creates  the  dialog  box  that  is  shown  to  the  user  when  asked  to  show 
'  the  current  weight  settings 

Sub  weightsetting_DialogFramel_Show() 

Sheets("Model").Activate 
With  ActiveDialog 

weightvalue  =  "RDTE  = "  &  Range("B3").Value 

.Labels("RDTE").Caption  =  weightvalue 

weightvalue  =  "Time  of  program  = "  &  Range("C3").  Value 

.Labels("time").Caption  =  weightvalue 

wei^tvalue  =  "IOC  = "  &  Range("D3").  Value 

.Labels("ioc").Caption  =  weightvalue 

weightvalue  =  "Development  cost  = "  &  Range("E3").Value 

.Labels("devel").Caption  =  weightvalue 

weightvalue  =  "O&S  cost  =  "  &  Range("F3").  Value 

.Labels("os").Caption  =  weightvalue 

weightvalue  =  "Production  cost  =  "  &  Range("G3"). Value 

.Labels("product").Caption  =  weightvalue 

weightvalue  =  "RCS  =  "  &  Range("H3").Value 

.LabelsC'rcs"). Caption  =  weightvalue 

weightvalue  =  "IR  signature  =  "  &  Range("I3").Value 

.Labels("ir").Caption  =  weightvalue 

weightvalue  =  "Empty  weight  =  "  &  Range("J3").  Value 

.Labels("ew").Caption  =  weightvalue 

weightvalue  =  "Payload  weight  =  "  &  Range("K3").  Value 

.Labels("payload").Caption  =  wdghtvalue 

weightvalue  =  "TOGW  =  "  &  Range("L3").Value 

.Labels("togw").Caption  =  weightvalue 

weightvalue  =  "Lift-dirty  =  "  &  Range("M3").  Value 

.Labels("ld").Caption  =  weightvalue 

weightvalue  =  "Lift-clean  = "  &  Range("N3").  Value 

.Labels("lc").Caption  =  weightvalue 

weightvalue  =  "Drag-sub-clean  = "  &  Range("03"). Value 

.Labels("dc").Caption  =  weightvalue 

wei^tvalue  =  "Drag-sub-dirty  =  "  &  Range("P3").  Value 
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.Labels("dd").Caption  =  wdghtvalue 
weightvalue  =  "Drag-super-clean  = "  &  Range("Q3").Value 
.Labels("dsc").Caption  =  weightvalue 
weightvalue  =  "Drag-super-dirty  =  "  &Range("R3").  Value 
.Labels("dsd").Caption  =  weightvalue 
weightvalue  =  "Thrust  = "  &  Range("S3").Value 
.Labels("thrust").Caption  =  weightvalue 
weightvalue  =  "Fuel  consumption  =  "  &  Range("T3").Value 
.Labels("fuer').Caption  =  weightvalue 
weightvalue  =  "Maneuveratdlity  = "  &  Range("U3").Value 
.LabelsC'man"). Caption  =  weightvalue 
End  With 
End  Sub 

'  the  next  three  are  the  preset  personality  types. 

'  to  change  the  preset  personalities  you  need  to  unhide  the  weightperson 
’  worksheet  and  then  change  the  appropriate  weight  of  the  sheet. 

'  Rehide  the  sheet  to  clean  up  the  work  environment 

'  perfonn_Click  Macro 
Sub  perform_ClickO 
weightperson  (3) 

End  Sub 

’  lab_Click  Macro 
Sub  lab_ClickO 
weightperson  (4) 

End  Sub 

'  overaIl_Click  Macro 
Sub  overall_ClickO 
weigh^erson  (5) 

End  Sub 

’  this  is  the  subroutine  that  actually  changes  the  weights  for  the  preset 
’  personality  types. 

Sub  weightperson(typeperson) 

Sheets("weigh^erson").Activate 

'impacts 

Range("  Weights!  $D$5").  Value  =  Cells(4,  typeperson) 

'rdte 

Range("Weights!$D$6").Value  =  Cells(5,  typeperson) 

'time  of  program 

Range("Weights!$D$7").  Value  =  Cells(6,  typeperson) 

'cost 

Range("Weights!$D$H").Value  =  Cells(7,  typeperson) 


72 


'ioc 

Range("Weights!$D$12").Value  =  Cells(8,  typeperson) 
'performance 

Range("Weights!$D$  13").  Value  =  Cells(9,  typeperson) 
'develop 

Range("Weights!$D$17").Value  =  CeUs(10,  typeperson) 
'o&s 

Range("Weights!$D$18").Value  =  Cells(ll,  typeperson) 
'production 

Range("Weights!$D$19").Value  =  Cells(12,  typeperson) 
'mass 

Range("Weights!$D$23").Value  =  Cells(13,  typeperson) 
'aero 

Range("Weights!$D$24").Value  =  Cells(14,  typeperson) 
'observability 

Range("Weights!$D$25").Value  =  Cells(15,  typeperson) 
'ew 

Range("Weights!$D$29").Value  =  Cells(16,  typeperson) 
'payload 

Range("Weights!$D$30").Value  =  CeUs(17,  typeperson) 
'togw 

Range("Weights!$D$31").Value  =  Cells(18,  typeperson) 
'lift 

Range("Weights!$D$35").Value  =  CeUs(19,  typeperson) 
'drag 

Range("Weights!$D$36").Value  =  CeUs(20,  typeperson) 
'thrust 

Range("Weights!$D$37").Value  =  Cells(21,  typeperson) 
'fuel 

Range("Weights!$D$38").Value  =  CeUs(22,  typeperson) 
'maneuv 

Range("Weights!$D$39").  Value  =  Cells(23,  typeperson) 
'res 

Range("Weights!$D$41").Value  =  Cells(24,  typeperson) 
'ir 

Range("Weights!$D$42").Value  =  Cells(25,  typeperson) 
'dirty 

Range("Weights!$D$47").Value  =  Cells(26,  typeperson) 
'clean 

Range("Weights!$D$48").Value  =  CeUs(27,  typeperson) 
'subsonic  clean 

Range("Weights!$D$53").Value  =  Cells(28,  typeperson) 
'subsonic  dirty 

Range("Weights!$D$54").Value  =  CeUs(29,  typeperson) 
'supersonic  clean 

Range("Weights!$D$55").Value  =  CeUs(30,  typeperson) 
'supersonic  dirty 
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Range("Weights!$D$56").Value  =  Cells(31,  typeperson) 
Woiksheets("Weights").Calculate 
End  Sub 

’  this  is  the  code  that  changes  the  weights  for  the  individual  that 
’  wants  to  personally  input  a  new  set  of  wei^ts 
'  All  the  weight  dialogboxes  must  be  called  in  order  to  correctly 
’  use  the  code.  The  weights  are  separated  by  6  rows. 

Sub  WeightsO 
Counter  =  5 

DialogSheets("Weightsr’).Show 
Counter  =  Counter  +  6 
DialogSheets("Weights2").Show 
Counter  =  6  +  Counter 
E)ialogSheets("Weights3").Show 
Counter  =  6  +  Counter 
DialogSheets("Weights4").Show 
Counter  =  6  +  Counter 
DialogSheets("Weights5").Show 
Counter  =  6  +  Coimter 
DialogSheets('’Weights6").Show 
Counter  =  6  +  Counter 
DialogSheets("Weights7").Show 
Counter  =  6  +  Counter 
DialogSheets("Weights8").Show 
Counter  =  6  +  Counter 
DialogSheets("Weights9").Show 
End  Sub 

'  weight_Show  Macro 

’  How  to  initialize  the  weights  assessment  box 
'  This  is  similar  to  the  evaluation  measures  initialization 

Sub  wei^t_ShowO 
Sheets("Weights").Activate 
With  ActiveDialog 
.EditBoxes.Text  =  "" 

.EditBoxes("textl").Text  =  Cells(Counter,  5) 
.EditBoxes("text2").Text  =  CeIls(Counter  +1,5) 
.EditBoxes("text3").Text  =  CeUs(Coimter  +  2, 5) 
.EditBoxes("text4").Text  =  Cells(Counter  +  3, 5) 
.EditBoxes("text5").Text  =  CeUs(Counter  +  4, 5) 
words  =  "Weight  of  "  &  Cells(Counter,  2) 
.Labels("labeir’).Caption  =  words 
words  =  "Weight  of "  &  Cells(Counter  +1,2) 
.Labels("label2").Caption  =  words 
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words  =  "Weight  of  "  &  CeUs(Counter  +  2, 2) 
.Labels("label3").Caption  =  words 
words  =  "Weight  of  "  &  Cells(Counter  +  3, 2) 
.Labels("label4").Caption  =  words 
words  =  "Weight  of "  &  Cells(Counter  +  4, 2) 
.Labels("label5").Caption  =  words 
End  With 
End  Sub 

’  NoButton_Click  Macro 

'  The  noButton  execution  for  the  evalmeasures  assessments 
’  This  is  done  for  each  evaluation  measure  dialog  box. 

'  It  allows  the  user  to  see  the  dialog  box  to  change  the  evaluation 
'  measure's  value  function 

Sub  NoButton_Qick() 

DialogSheets("EvalDialog").Show 
End  Sub 

'  Risk_NoButton_Qick  Macro 

I 

Sub  Risk_NoButton_QickO 

DialogSheets("EvalDialog").Show 
End  Sub 

'  IOC_NoButton_Qick  Macro 
! 

Sub  IOC_NoButtDn_ClickO 
DialogSheets("EvalDialog").Show 
End  Sub 

’  tLmeo^rogram_NoButton_Click  Macro 

I 

Sub  iimeoQ)rogram_NoButton_Click() 
DialogSheets("EvalDialog").Show 
End  Sub 

’  YesButton_Qick  Macro 

I 

Sub  YesButton_Click() 

End  Sub 

’  Develop_NoButton_Qick  Macro 

Sub  Develop_NoButton_QickO 
DialogSheets("Evall>ialog").Show 
End  Sub 


75 


’  OS_NoButton_Click  Macro 

I 

Sub  OS_NoButton_ClickO 
DialogSheets("EvalE>ialog").Show 
End  Sub 

t 

’  Product_NoButton_Qick  Macro 
» 

Sub  Product_NoButton_Click() 
DialogSheets("EvalDialog").Show 
End  Sub 

t 

’  RCS_NoButton_Click  Macro 

t 

Sub  RCS_NoButton_ClickO 
DialogSheets("EvalDialog").Show 
End  Sub 

t 

'  IR_NoButton_Click  Macro 

t 

Sub  IR_NoButton_QickO 
DialogSheets("EvalDialog").Show 
End  Sub 

I 

’  EW_NoButton_Qick  Macro 

I 

Sub  EW_NoButton_aickO 
DialogSheets("EvalDialog").Show 
End  Sub 

t 

’  Payload_NoButton_Qick  Macro 

t 

Sub  Payload_NoButton_ClickO 
DialogSheets("EvalI>ialog").Show 
End  Sub 

t 

'  T(X}W_NoButton_Click  Macro 
! 

Sub  TOGW_NoButton_ClickO 
DialogSheets("EvalDialog").Show 
End  Sub 

t 

'  Dirty_NoBulton_Click  Macro 

I 

Sub  Dirty_NoButton_Click() 

E)ialogSheets("EvalDialog").Show 


End  Sub 


'  Clean_NoButton_Click  Macro 

Sub  Qean_NoButton_Qick() 
I>ialogSheets('EvalDialog").Show 
End  Sub 

t 

'  Sub_NoButton_Click  Macro 

f 

Sub  Sub_NoButton_C!lickO 
DialogSheets("EvalDialog").Show 
End  Sub 

’  SuD_NoButton_Click  Macro 

t 

Sub  SuD_NoButton_Click() 
DialogSheets("EvalDialog").Show 
End  Sub 

t 

’  Super_NoButton_Click  Macro 

t 

Sub  Super_NoButton_Click() 

DialogSheets('EvalDialog").Show 

End  Sub 
! 

’  SuperD_NoButton_Qick  Macro 

I 

Sub  SuperD_NoButton_Qick() 
DialogSheets("EvalDialog").Show 
End  Sub 

I 

’  Thrust_NoButton_Click  Macro 

I 

Sub  Thrust_NoButton_Qick() 
DialogSheets(’'EvalDialog").Show 
End  Sub 

’  Fuel_NoButton_Click  Macro 

Sub  Fuel_NoButton_Qick() 
DialogSheets("EvalDialog").Show 
End  Sub 

’  Maneuv_Button3_Qick  Macro 
'  This  is  the  No  Button 
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Sub  Maneuv_Button3_QickO 
DialogSheets("EvalDialog").Show 
End  Sub 

'the  nobutton  info  for  the  weights  assessment 
'  Button3_Click  Macro 
'  this  is  for  weightsl 

f 

Sub  Button3_Click() 

DialogSheets("WeightDialog").Show 
End  Sub 

t 

'  Weights2_Button3_C!lick  Macro 

t 

Sub  Weights2_Button3_Click() 

DialogSheets("WeightDialog").Show 
End  Sub 

t 

'  Weights3_NoButton_Click  Macro 

Sub  Weights3_NoButton_ClickO 
DialogSheets("WeightDialog").Show 
End  Sub 

t 

'  Weights4_NoButton_Click  Macro 

I 

Sub  Weights4_NoButton_ClickO 
E)ialogSheets("WeightDialog").Show 
End  Sub 

t 

'  Weights5_NoButton_Click  Macro 
! 

Sub  Weights5_NoButton_ClickO 

DialogSheets("WeightDialog").Show 

End  Sub 
! 

'  Weights6_NoButton_Click  Macro 

I 

Sub  Weights6_NoButton_ClickO 
DialogSheets("WeightDialog").Show 
End  Sub 

I 

’  Weights7_Button3_C3ick  Macro 

I 

Sub  Weights7_Button3_Qick() 
DialogSheets("WeightDialog").Show 
End  Sub 


’  Weights8_Button3_Click  Macro 

I 

Sub  Weights8_Button3_aick() 

DialogSheets("WeightDialog").Show 
End  Sub 


'  Dialog27_Button3_Click  Macro 
’  Actually  refers  to  weights9 

Sub  Dialog27_Button3_QickO 
DialogSheets("WeightDialog").Show 
End  Sub 

'  Button2_Click  Macro 

'  This  gets  the  sensitivity  analysis  working 

'  There  are  different  versions  woridng  to  save  code  space;  however, 

'  the  code  used  for  5  way  sensitivity  analysis  can  be  used  for  all 
’  of  the  versions,  with  only  the  first  ones  being  called  for  use. 

Sub  Butt(Mi2_C!lickO 
Num  =  Range("Model!$A$l").Value 
Sheets("Weights")-Activate 
Range(Cells(68, 3),  Cells(79, 13)).Select 
Selection.Copy 

Selection.PasteSpecialPaste:=xlValues,  Operation:=xlNone,  _ 
SkipBlanks:=False,  Transpose  :=False 

*3  item  sensitivity  analysis  weights  1 

If  DialogSheets("SenAnal").OptionButtons("impacts’’).Value  =  xlOn  TTien 
Range("F5").Select 
ActiveCell.FormulaRlCl  =  "=RC[-1]’’ 

Range("F6").Select 
ActiveCeU-FormulaRlCl  =_ 

"=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[2]C[-1])" 
Range("F7").Select 
ActiveCell.FonnulaRlCl  =  _ 

"=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F8’’).Select 
ActiveCell.FonnulaRlCl  =  _ 

"=(1-R[-3]C)*(RC[-1])/(R[-2]C[-1]+R[-1]C[-1]+RC[-1])" 
Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 
Selection.TableRowInput:=Range("F5") 
titlename  =  "Aircraft  Impacts" 
vertline  =  "F5" 

ElseIfDialogSheets("SenAnal").OptionButtons("RDTE").  Value  =  xlOn  Then 
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Range("F6").Select 
ActiveCeUPonnulaRlCl  =  "=RC[-1]" 

Range("F5").Select 
ActiveCell-FormiilaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(RC[-1]+R[2]C[-1]+R[3]C[-1])" 
Range("F7").Select 
ActiveCell.FomiulaRlCl  =  _ 

"=(1-R[-1]C)*(RC[-1])/(R[-2]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F8").Select 
ActiveCell.FormxilaRlCl  =  _ 

"=(1-R[-2]C)*(RC[-1])/(R[-3]C[-1]+R[-1]C[-1]+RC[-1])" 
Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 
Selection.Table  RowInput;=Range("F6") 
titlename  =  "Lab  RDTE  Cost" 
vertline  =  "F6" 

Elself  DialogSheets("SenAnal").OptionButtons("prograin").Value  =  xlOnTTien 
Range("F7").Select 
ActiveCeUPonnulaRlCl  =  "=RC[-1]" 

Range("F5").Select 
ActiveCeUPonnulaRlCl  =_ 

"=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[3]C[-1])" 
Range("F6").Select 
ActiveCeUPonnulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[2]C[-1])" 
Range("F8").Select 
ActiveCeUPonnulaRlCl  =  _ 

"=(1-R[-1]C)*(RC[-1])/(R[-3]C[-1]+R[-2]C[-1]+RC[-1])" 
Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 
Selection.TableRowInput:=Range("F7") 
titlename  =  "Time  of  Program" 
vertline  =  ’P7" 

'3  item  sensitivity  analysis  Weights  2 

Elself  DialogSheets("SenAnal").OptionButtons("cost").Value  =  xlOn  Then 
Range("Fll").Select 
ActiveCeUPonnulaRlCl  =  "=RC[-1]" 

Range("F12").Select 

ActiveCeUPonnulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F13").Select 

ActiveCeUPonnulaRlCl  =  "=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("Fl  1 ") 

titlename  =  "Cost" 

vertline  =  Til" 

Elself  DialogSheets("SenAnal").OptionButtons("IOC").Value  =  xlOn  Then 
Range("F12").Select 
ActiveCeUPonnulaRlCl  =  "=RC[-1]" 
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RangeC'Fl  l”).Select 

ActiveCeU.FonnulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[2]C[-1]+RC[-1])" 
RangeC'F13").Select 

ActiveCeU.FonnulaRlCl  =  "=(l-R[-l]C)*(RC[-l])/(RC[-l]+R[-2]C[-l])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("F12") 

titlename  =  "IOC" 

verfUne  =  "F12" 

Elself  DialogSheets("SenAnal").OptionButtons("perfonn").Value  =  xlOn  Then 
Range("F13").Select 
ActiveCeU.FomiulaRlCl  =  "=RC[-1]" 

RangeC'FH  ").Select 

ActiveCeU.FonnulaRlCl  =  "=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F12").Select 

ActiveCeU-FoimulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("F13") 

titlename  =  Terfonnance" 

vertline  =  "F13" 


*3  item  sensitivity  analysis  Weights  3 

Elself  DialogSheets("SenAnal").OptionButtons("develop").  Value  =  xlOn  Then 
Range("F17").Select 
ActiveCell.FonnulaRlCl  =  "=RC[-1]" 

RangeC'Fl  8").Select 

ActiveCell-FomiulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F19").Select 

ActiveCeUPoimulaRlCl  =  "=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(Cells(68,  2),  Cells(68  +  Num,  13)).Select 

Selection.TableRowInput:=Range("F17") 

titlename  =  "Development  Cost" 

vertline  =  "F17" 

Elself  DialogSheets("SenAnal").OptionButtons("OS").  Value  =  xlOn  Then 
RangeC'Fl  8").Select 
ActiveCell.FormulaRlCl  =  "=RC[-1]" 

Range("F17").Select 

ActiveCell-FormulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[2]C[-1]+RC[-1])" 
Range("F19").Select 

ActiveCell.FormulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[-2]C[-1])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.TableRowInput:=Range("F18") 

titlename  =  "O&S  Cost" 

vemine  =  "F18" 

Elself  DialogSheets("SenAnal").C)ptionButtons(”product").  Value  =  xlOnThen 
Range("F19").Select 
ActiveCell.FonnulaRlCl  =  "=RC[-1]" 

Range("F17").Select 
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ActiveCeU.FonnulaRlCl  =  "=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F18’').Select 

ActiveCeU-FormulaRlCl  =  ’'=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.TableRowInput:=Range("F19") 

tidename  =  "Production  Cost" 

vertiine  =  "F19" 


'3  item  sensitivity  analysis  Weights  4 

Elself  DialogSheets("SenAnal").C)ptionButtons("mass").Value  =  xlOn  Then 
Range("F23").Select 
ActiveCell.FonnulaRlCl  =  "=RC[-1]" 

Range("F24").Select 

ActiveCell.FormulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F25").Select 

ActiveCell.FormulaRlCl  =  "=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.Tahle  RowInput:=Range("F23") 

titiename  =  "Mass" 

vertiine  =  "F23" 

Elself  DialogSheets("SenAnal").OptionButtons("aeio"),  Value  =  xlOn  Then 
Range("F24").Select 
ActiveCell.FormulaRlCl  =  "=RC[-1]" 

Range("F23").Select 

ActiveCell.FonnulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[2]C[-1]+RC[-1])" 
Range("F25").Select 

ActiveCeU-ForaiulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[-2]C[-1])" 

Range(CeUs(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.Tahle  RowInput:=Range("F24") 

titiename  =  "Aero" 

vertiine  =  "F24" 

Elself  DialogSheets("SenAnal").C)ptionButtons("ohservahility").Value  =  xlOnThen 
Range("F25").Select 
ActiveCeRFormulaRlCl  =  "=RC[-1]" 

Range("F23").Select 

ActiveCeU.FormulaRlCl  =  "=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1])” 
Range("F24").Select 

ActiveCell.FonnulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.TahleRowInput:=Range("F25") 

titiename  =  "Ohservahility" 

vertiine  =  "F25" 


'2  item  sensitivity  analysis  weights  5 

Elself  DialogSheets("SenAnal").C)ptionButtons("EW").  Value  =  xlOn  Then 
Range("F29").Select 
ActiveCeEFoimulaRlCl  =  "=RC[-1]" 
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Range("F30").Select 

ActiveCeU.FonnulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F3  r’).Select 

ActiveCeUPoraiulaRlCl  =  ’’=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 

Selection.TableRowInput:=Range("F29") 

tidename  =  "Empty  Weight" 

verdine  =  "F29" 

ElseIfDialogSheets("SenAnal").C)ptionButtons("payload").  Value  =  xlOn  Then 
Range("F30").Select 
ActiveCeRFormulaRlCl  =  "=RC[-1]" 

Range("F29").Select 

ActiveCeRFormulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[2]C[-1]+RC[-1])" 
Range("F31").Select 

ActiveCeU.FonnulaRlCl  =  "=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[-2]C[-1])" 

Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("F30") 

ddename  =  "Payload  Weight" 

vertline  =  "F30" 

ElseIfE)ialogSheets("SenAnal").C)ptionButtons("TOGW").  Value  =  xlOn  Then 
Range("F31").Select 
ActiveCell-FonnulaRlCl  =  "=RC[-1]" 

Range("F29").Select 

ActiveCell.ForaiulaRlCl  =  "=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1])" 
Range("F30").Select 

ActiveCeU.FonnulaRlCl  =  "=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1])" 

Range(CeUs(68,  2),  Cells(68  +  Num,  13)).Select 

Selection.Table  RowInput;=Range("F31") 

tidename  =  "Observability" 

verdine  =  "F31" 


'5  item  sensitivity  analysis  weights  6 

ElseIfDialogSheets("SenAnal").OptionButtons("lift").  Value  =  xlOn  Then 
Range("F35").Select 
ActiveCell-FormulaRlCl  =  "=RC[-1]" 

Range("F36").Select 
ActiveCell.FonnulaRlCl  =  _ 

"=(1-R[-1]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[2]C[-1]+R[3]C[-1])" 
Range("F37").Select 
ActiveCell-FormulaRlCl  =_ 

"=(1-R[-2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[1]C[-1]+R[2]C[-1])" 
Range("F38").Select 
ActiveCeU.FormulaRlCl  =  _ 

"=(1-R[-3]C)*CRC[-1])/(R[-2]C[-1]+R[-1]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F39").Select 
ActiveCell-FormidaRlCl  =  _ 

"=(1-R[-4]C)*(RC[-1])/(R[-3]C[-1]+R[-2]C[-1]+R[-1]C[-1]+RC[-1])" 
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Range(CeUs(68, 2),  Cells(68  +  Num,  13)).Select 
Selection.TableRowInput:=Range("F35") 
titlename  =  "lift" 
vertline  =  "F35" 

ElseIfI>ialogSheets("SenAnal").C)ptionButtons("drag").  Value  =  xlOnThen 
Range("F36").Select 
ActiveCell-FormulaRlCl  =  "=RC[-1]" 

Range("F35").Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(RC[-1]+R[2]C[-1]+R[3]C[-1]+R[4]C[-1])" 
Range("F37").Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[-1]C)*(RC[-1])/(R[-2]C[-1]+RC[-1]+R[1]C[-1]+R[2]C[-1])" 
Range("F38")-Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[-2]C)*(RC[-1])/(R[-3]C[-1]+R[-1]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F39").Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[-3]C)*(RC[-1])/(R[-4]C[-1]+R[-2]C[-1]+R[-1]C[-1]+RC[-1])" 
Range(Cells(68, 2),  Cells(68  +  Num,  13))-Select 
Selection-Table  RowInput:=Range("F36") 
tiflename  =  "Drag" 
verfline  =  "F36" 

Elself  DialogSheets("SenAnal")-C)ptionButtons("thrust")-Value  =  xlOn  Then 
Range("F37")-Select 
ActiveCell-FormulaRlCl  =  "=RC[-1]" 

Range("F35")-Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[2]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[3]C[-1]+R[4]C[-1])" 
Range("F36")-Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[2]C[-1]+R[3]C[-1])" 
Range("F38")-Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[-1]C)*(RC[-1])/(R[-3]C[-1]+R[-2]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F39")-Select 
ActiveCell-FormulaRlCl  =  _ 

"=(l-R[-2]C)*(RC[-l])/(R[-4]C[-l]+R[-3]C[-l]+R[-l]C[-l]+RC[-l])" 
Range(Cells(68, 2),  Cells(68  +  Num,  13))-Select 
Selection-Table  RowInput:=Range("F37") 
titiename  =  "Thrust" 
verfline  =  "F37" 

Elself  DialogSheets("SenAnal")-C)ptionButtons("fuer’)-Value  =  xlOn  Then 
Range("F38").Select 
ActiveCeU-FonmflaRlCl  =  "=RC[-1]" 

Range("F35")-Select 
ActiveCell-FormulaRlCl  =  _ 
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"=(1-R[3]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[2]C[-1]+R[4]C[-1])" 
Range("F36").Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[1]C[-1]+R[3]C[-1])" 
Range("F37").Select 
ActiveCellPormulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(R[-2]C[-1]+R[-1]C[-1]+RC[-1]+R[2]C[-1])" 
Range("F39").Select 
ActiveCell-ForamlaRlCl  =  _ 

"=(l-R[-l]C)*(RC[-l])/(R[4]C[-l]+R[-3]C[-l]+R[-2]C[-l]+RC[-l]r 
Range(CeUs(68, 2),  CeUs(68  +  Num,  13)).Select 
Selection.Table  RowInput:=Range("F38") 
tillename  =  "Fuer’ 
veriline  =  "F38" 

ElseIfDialogSheets("SenAnal").C)plionButtons("maneuv").  Value  =  xlOnThen 
Range("F39").Select 
ActiveCell.FormulaRlCl  =  "=RC[-1]" 

Range("F35").Select 
ActiveCeU.FormulaRlCl  =  _ 

"=(1-R[4]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[2]C[-1]+R[3]C[-1])" 
Range("F36").Select 
ActiveCell.FormulaRlCl  =_ 

"=(1-R[3]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[1]C[-1]+R[2]C[-1])" 
Range(’’F37").Select 
ActiveCellPormulaRlCl  =  _ 

"=(1-R[2]C)*(RC[-1])/(R[-2]C[-1]+R[-1]C[-1]+RC[-1]+R[1]C[-1])" 
Range("F38").Select 
ActiveCell-FormulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(R[-3]C[-1]+R[-2]C[-1]+R[-1]C[-1]+RC[-1])" 
Range(Cells(68, 2),  CeUs(68  +  Num,  13)).Select 
Selection.Table  RowInput:=Range("F38 ") 
titiename  =  "Maneuverability" 
vertline  =  "F39" 

'2  item  sensitivity  analysis  weights  7 

ElseIfDialogSheets("SenAnal").OptionButtons('RCS").  Value  =  xlOn  Then 
Range(Cells(68,  2),  Cells(68  +  Num,  13)).Select 
Selection.Table  RowInput:=Range(T41 ") 
titiename  =  "RCS  (IR  signature  is  the  complement)" 
vertline  =  "F41" 

'2  item  sensitivity  analysis  weights_8 

ElseIfDialogSheets("SenAnal").OptionButtons("dirty").  Value  =  xlOn  Then 
Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 
Selection.TableRowInput:=Range("F47") 
titiename  =  "Lift  dirty  (Lift  clean  is  the  complement)" 
vertline  =  "F47" 
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'4  item  sensitivity  analysis  weights  9 

Elself  DialogSheets("SenAnal").OptionButtons("sub").  Value  =  xlOn  Then 
Range("F53’').Select 
ActiveCell.FoimulaRlCl  =  "=RC[-1]" 

Range("F54").Select 

ActiveCell.FonnulaRlCl  =  "=(l-R[-l]C)*(RC[-l])/(RC[-l]+R[l]C[-l]+R[2]C[-l])" 

Range("F55").Select 

ActiveCell.FonnulaRlCl  =  "=(l-R[-2]C)*(RC[-l])/(R[-l]C[-l]+RC[-l]+R[l]C[-l])" 
Range("F56").Select 

ActiveCell.FonnulaRlCl  =  "=(l-R[-3]C)*(RC[-l])/(R[-2]C[-l]+R[-l]C[-l]+RC[-l])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("F53") 

titlename  =  "Subsonic  Drag  Qean" 

vertline  =  "F53" 

Elself  DialogSheets("SenAnal").C)ptionButtons("subd").  Value  =  xlOn  Then 
Range("F54").Select 
ActiveCeU-FormulaRlCl  =  "=RC[-1]" 

Range("F53").Select 

ActiveCell.FoimulaRlCl  =  "=(l-R[l]C)*(RC[-l])/(RC[-l]+R[2]C[-l]+R[3]C[-l])" 

Range("F55").Select 

ActiveCell.FomiulaRlCl  =  "=(l-R[-l]C)*(RC[-l])/(R[-2]C[-l]+RC[-l]+R[l]C[-l])" 

Range("F56").Select 

ActiveCeU.FonnulaRlCl  =  "=(l-R[-2]C)*(RC[-l])/(R[-3]C[-l]+R[-l]C[-l]+RC[-l])" 

Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 

Selection.TableRowInput:=Range("F54") 

titlename  =  "Subsonic  Drag  Dirty" 

vertUne  =  "F54" 

Elself  DialogSheets("SenAnal").OptionButtons("  super").  Value  =  xlOn  Then 
Range("F55").Select 
ActiveCeU.FonnulaRlCl  =  "=RC[-1]" 

Range("F53").Select 

AcaveCell.FonnulaRlCl  =  "=(l-R[2]C)*(RC[-l])/(RC[-l]+R[l]C[-l]+R[3]C[-l])" 
Range("F54").Select 

ActiveCeU.FonnulaRlCl  =  "=(l-R[l]C)*(RC[-l])/(R[-l]C[-l]+RC[-l]+R[2]C[-l])" 
Range("F56").Select 

ActiveCell.FomiulaRlCl  =  "=(l-R[-l]C)*(RC[-l])/(R[-3]C[-l]+R[-2]C[-l]+RC[-l])" 

Range(Cells(68, 2),  CeUs(68  +  Num,  13)).Select 

Selection.Table  RowInput:=Range("F55") 

tiflename  =  "Supersonic  Drag  Qean" 

vertline  =  "F55" 

Elself  DialogSheets("SenAnal").C)ptionButtons("superd").  _ 

Value  =  xlOn  Then 
Range("F56").Select 
ActiveCeU.FormulaRlCl  =  "=RC[-1]" 

Range("F53").Select 
ActiveCell.FormulaRlCl  =_ 
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"=(1-R[3]C)*(RC[-1])/(RC[-1]+R[1]C[-1]+R[2]C[-1])’’ 
Range("F54’').Select 
ActiveCell.FonnulaRlCl  =  _ 

"=(1-R[2]C)*(RC[-1])/(R[-1]C[-1]+RC[-1]+R[1]C[-1])’' 
Range("F55").Select 
ActiveCellPormulaRlCl  =  _ 

"=(1-R[1]C)*(RC[-1])/(R[-2]C[-1]+R[-1]C[-1]+RC[-1])" 
Range(Cells(68, 2),  Cells(68  +  Num,  13)).Select 
Selection.Table  RowInput:=Range("F56") 
titlename  =  "Supersonic  Drag  Dirty" 
vertline  =  "F56" 

End  If 

'  now  the  sensitivity  fiinctions  have  been  updated  the  old  chart  must  be 
’  deleted  then  a  new  chart  created  with  the  proper  title 
’  Then  the  chart  must  be  copied  to  the  dialog  box  that  is  shown 
’  to  the  user.  This  cut  and  paste  properties  are  different 
’in  office  97.  Then  the  dialog  box  is  shown. 
ActiveSheet.ChartObjects("sensit").Delete 
current  =  Range(vertlme).  Value 

tidetext  =  "Weight  of "  &  titlename  &  "  currrently  "  &  current 
ActiveSheet.ChartObjects.Add(45, 897, 346.5, 194.25).Select 
Calculate 

ActiveChartChartWizard  Source:=Range(Cells(80, 2),  _ 

Cells(80  +  Num,  1 3)),  Gallery  ;=  _ 
xlLine,  Format:=2,  PlotBy:=xlRows,  CategoryLabels:=l,  _ 
SeriesLabels:=l,  HasLegend:=l,  Title:="",  Category'ntle:=  _ 
titletext,  ValueTitle:="Value",  ExtraTitle:="" 

Selection.Name  =  "sensit" 

Sheets("sendialog").Select 
ActiveSheet.DrawingObjects("sensit").Select 
Selection.Delete 
Sheets("Weights").Select 
ActiveSheet.DrawingObjects("sensit").Select 
Selection.Copy 
Sheets("sen(fi£dog").Select 
ActiveSheetPaste 
Selection.Left  =  73.5 
Selection.Top  =  36.75 
Sheets("Weights").Select 
DialogSheets("sendialog").Show 
End  Sub 


'  DataEntryStart_Click  Macro 
'  show_Click  Macro 

'  This  does  the  same  type  of  thing  as  the  previous  code  however  it  waits 
'  to  find  out  the  type  of  evaluation  the  user  wants  then  creates  the 
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'  chart  as  requested. 


Sub  show_QickO 

Num  =  Range("Model!$A$l").  Value 
Sheets("Model").Activate 
ActiveSheet.ChartObjects('’dataeval").Delete 
If  DialogSheets("dialogeval").C)ptionButtons("bar").  _ 

Value  =  xlOn  Then 

ActiveSheet.ChartObjects.Add(45, 667,  346.5, 194.25).Select 

Calculate 

ActiveChaitChartWizard  _ 

Source:=Sheets("Moder’).Range(CeUs(60, 1),  _ 

Cells(60  +  Num,  2)),  Gallery:=xlColumn,  _ 

Format:=3,  PlotBy:=xlColumns,  CategoryLabels:=l,  _ 
SeriesLabels:=l,  HasLegend:=l,  Title:="",  _ 
CategoryTitle:="Technology",  ValueTitle:="Value",  _ 
ExtraTitle:="" 

Elself  DialogSheets("dialogeval").OptionButtons("main").  _ 

Value  =  xlOn  Then 

ActiveSheet.ChartObjects.Add(45, 667,  346.5, 194.25).Select 

Calculate 

ActiveChart.ChartWizard  _ 

Source:=Sheets("Model").Range(Cells(48, 1),  _ 

Cells(48  +  Num,  8)),  Gallery:=xlColumn,  _ 

Format:=3,  PlotBy:=xlColumns,  CategoryLabels:=l ,  _ 
SeriesLabels:=l,  HasLegend;=l,  Title:="",  _ 
CategoryTitle:="Technology",  _ 

ValueTitle:="Value’',  ExtraTitle:="" 

Else 

ActiveSheet.ChartObjects.Add(45, 667, 346.5, 194.25).Select 

Calculate 

ActiveChart.ChartWizard  _ 

Source:=Sheets("Moder').Range(Cells(36, 1),  _ 

Cells(36  +  Num,  21)),  GaIlery:=xlColumn,  _ 

Format:=3,  PlotBy:=xlColumns,  CategoryLabels:=l,  _ 
SeriesLabels:=l,  HasLegend:=l,  Title:="",  _ 
CategoryTitle:="Technology",  _ 

ValueTitle:='’Value",  ExtraTitle:="" 

Endlf 

Selection.Name  =  "dataeval" 

Sheets("datachart").Select 

ActiveSheet.DrawingObjects("dataeval").Select 

Selection.Delete 

Sheets("Moder’).Select 

ActiveSheet.DrawingObjects("dataeval").Select 

Selection.Copy 

Sheets("datachart").Select 
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ActiveSheetPaste 
Selection.Left  =  73.5 
Selection.Top  =  36.75 
Sheets("Moder').Select 
DialogSheets("datachart").Show 
End  Sub 


’  DialogFramel_Show  Macro 
’  for  uncertainty  dialog  sheet 

'  This  causes  the  uncertainty  analysis  dialog  box  be  empty 
’  when  the  user  calls  it  up. 

Sub  DialogFramel_ShowO 
With  ActiveEfialog 
.QieckBoxes.  Value  =  False 
End  With 
End  Sub 


'  Uncertainty_okbutton_Qick  Macro 
’  This  counts  the  number  of  uncertain  technologies  then  calls 
’  different  type  of  analysis  procedure  to  be  called  for  the  number 
'  of  uncertain  variables.  Then  it  calls  the  procedure  to  create  the 
’  CDF  for  the  data. 

'  The  variable  "place"  shifts  the  recorded  location  for  each  new 
'  uncertain  evaluation  measure.  :Number:  is  the  count  for  the  number 
’  of  imcertain  variables.  Only  the  first  five  uncertain  variables 
'  will  be  used 

Sub  Uncertainty_okbutton_ClickO 
Num  =  Range("Model!$A$l").Value 
SheetsC’data").  Activate 
number  =0 
place  =  2 
rownum  =  102 

If  DialogSheets("Uncertainty").CheckBoxes("boxl").  Value  =  _ 
xlOn  Then 

number  =  number  +1 
Cellsfrowmun,  place)  =  1 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box2").  Value  =  _ 
xlOnThen 

number  =  number  +  1 
Cells(rownum,  place)  =  2 
place  =  place  +1 
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End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box3").  Value  =  _ 
xlOnThen 

number  =  number  + 1 
CeUs(rownum,  place)  =  3 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box4").Value  =  _ 
xlOnThen 

number  =  munber  +  1 
Cells(rownum,  place)  =  4 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box5").Value  =  _ 
xlOn  Then 

munber  =  number  +  1 
Cells(rownum,  place)  =  5 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box6").Value  =  _ 
xlOnThen 

number  =  number  +  1 
CeUs(rownum,  place)  =  6 
place  =  place  +  1 
End  If 

If  DialogSheets("Unceitainty").CheckBoxes("box7").  Value  =  _ 
xlOn  Then 

number  =  munber  +  1 
Cells(rowraun,  place)  =  7 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box8").Value  =  _ 
xlOnThen 

number  =  number  +1 
Cells(rownum,  place)  =  8 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box9").Value  =  _ 
xlOn  Then 

munber  =  number  +  1 
Cells(rownum,  place)  =  9 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("boxlO").Value  =  _ 
xlOnThen 

munber  =  number  +  1 
CeIls(rownum,  place)  =  10 
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place  =  place  +  1 
End  If 

If  DialogSheets("Uncertamty").CheckBoxes("boxir’).Value  =  _ 
xIOnThen 

number  =  number  +  1 
CelIs(rownum,  place)  =  11 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("boxl2").VaIue  =  _ 
xlOn  Then 

number  =  number  +  1 
CelIs(rownum,  place)  =  12 
place  =  place  +1 
End  If 

If  DialogSheets("Unceitainty").QieckBoxes("boxl3").VaIue  =  _ 
xlOn  Then 

number  =  number  +  1 
Cells(rownum,  place)  =13 
place  =  place  +1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("boxl4").VaIue  =  _ 
xlOn  Then 

number  =  number  +  1 
CelIs(rownum,  place)  =  14 
place  =  place  +  1 
End  If 

If  DiaIogSheets("Uncertainty").CheckBoxes("boxl5").  Value  =  _ 
xlOn  Then 

number  =  number  +1 
Cells(rownum,  place)  =  15 
place  =  place  +1 
End  If 

If  DiaIogSheets("Unceitainty").CheckBoxes("boxl6").  Value  =  _ 
xlOnThen 

number  =  number  +1 
Cells(rownum,  place)  =  16 
place  =  place  +  1 
End  If 

If  E)iaIogSheets("Unceitainty").CheckBoxes("boxl7").VaIue  =  _ 
xlOn  Then 

number  =  number  +  1 
CelIs(rownum,  place)  =  17 
place  =  place  +  1 
End  If 

If  DiaIogSheets("Uncertainty").CheckBoxes("boxl8").VaIue  =  _ 
xlOn  Then 

number  =  munber  +  1 


91 


Cells(rQwnum,  place)  =  18 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("boxl9").Value  = 
xlOnThen 

number  =  number  +  1 
CeUs(rownum,  place)  =  19 
place  =  place  +  1 
End  If 

If  DialogSheets("Uncertainty").CheckBoxes("box20").Value  = 
xlOnThen 

number  =  number  +  1 
Cells(rownum,  place)  =  20 
place  =  place  +  1 
End  If 

If  number  >  5  Then 

MsgBox  ("Only  the  first  5  uncertain  events  will  be  used") 
Uncertainty_Anal5 
Endlf 

If  number  =  1  Then 
Uncertainty_Anall 
Endlf 

If  number  =  2  Then 

MsgBox  ("Please  allow  the  computer  a  few  minutes.") 
Uncertainty_Anal2 
Endlf 

If  number  =  3  Then 

MsgBox  ("Please  allow  the  computer  a  few  minutes.") 
Uncertainty_Anal3 
Endlf 

If  number  =  4  Then 

MsgBox  ('Please  allow  the  computer  up  to  5  minutes.") 
Uncertainty_Anal4 
Endlf 

If  number  =  5  Then 

MsgBox  ("Please  allow  the  computer  up  to  20  minutes.") 
Uncertainty_Anal5 
Endlf 
cdf 

End  Sub 


’  This  is  the  uncertainty  analysis  for  only  one  uncertain  variable 
'  The  worksheet  is  calculated  3  times 

Sub  Uncertainty_AnallO 
Num  =  Range("Model!$A$l").Value 
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Sheets("data").Activate 
rowindex  =  105 
For  acount  =  1  To  3 
per  =  1 

per  =  percentage(acoimt,  per) 
evall  =  Cells(102, 2) 

For  techn  =  1  To  Num 
Cells(4  +  evall,  techn  +  1)  =  _ 

Cells(4  +  evall  +  acount  *  25,  techn  + 1) 
Next  techn 
Calculate 

CeUs(rowindex,  1)  =  per 
For  techn  =  1  To  Num 
CeUs(rowindex,  techn  +  1)  =  _ 

CeUs(104,  techn +1) 

Next  techn 

rowindex  =  rowindex  +  1 
Next  acount 

Cells(103, 2)  =  rowindex  - 105 
End  Sub 


'  This  is  the  imcertainty  analysis  for  two  uncertain  variables 
'  The  woricsheet  is  calculated  9  times 

Sub  Uncertainty_Anal20 
Num  =  Range("Model!$A$l").  Value 
Sheets("data").Activate 
rowindex  =  105 
For  acount  =  1  To  3 
evall  =  Cells(102, 2) 

For  techn  =  1  To  Num 
CeUs(4  +  evall,  techn  +  1)  =  _ 

Cells(4  +  evall  +  acount  *  25,  techn  +1) 

Next  techn 
For  bcotmt  =  1  To  3 
per  =1 

per  =  percentage(acount,  per) 
per  =  percentage(bcount,  per) 
eval2  =  CeUs(102, 3) 

For  techn  =  1  To  Num 
Cells(4  +  eval2,  techn  +  1)  =  _ 

Cells(4  +  eval2  +  bcount  *  25,  techn  + 1) 
Next  techn 
Calculate 

Cells(rowindex,  1)  =  per 
For  techn  =  1  To  Num 
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Cells(rowindex,  techn  + 1)  =  _ 
Cells(104,  techn  + 1) 

Next  techn 

rowindex  =  rowindex  +  1 
Next  bcount 
Next  acount 

Cells(103, 2)  =  rowindex  - 105 
End  Sub 


’  This  is  the  uncertainty  analysis  for  three  uncertain  variables 
'  The  worksheet  is  calculated  27  times 

Sub  Uncertainty_Anal3() 

Num  =  Range("Model!$A$l").Value 
Sheets("data").Activate 
rowindex  =  105 
For  acount  =  1  To  3 
evall  =  Cells(102, 2) 

For  techn  =  1  To  Num 
Cells(4  +  evall,  techn  +  1)  =  _ 

Cells(4  +  evall  +  acount  *  25,  techn  +  1) 

Next  techn 
For  bcount  =  1  To  3 
eval2  =  Cells(102, 3) 

For  techn  =  1  To  Num 
Cells(4  +  eval2,  techn  +  1)  =  _ 

Cells(4  +  eval2  +  bcount  *  25,  techn  +  1) 

Next  techn 
For  ccount  =  1  To  3 
eval3  =  CeUs(102, 4) 

For  techn  =  1  To  Num 
Cells(4  +  eval3,  techn  +  1)  =  _ 

Cells(4  +  eval3  +  ccount  *  25,  techn  + 1) 
Next  techn 
per  =  1 

per  =  percentage(acoimt,  per) 
per  =  percentage(bcount,  per) 
per  =  percentage(ccount,  per) 

Calculate 

Cells(rowindex,  1)  =  per 
For  techn  =  1  To  Num 

Cells(rowindex,  techn  +  1)  =  _ 

Cells(104,  techn  + 1) 

Next  techn 

rowindex  =  rowindex  +  1 
Next  ccount 
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Next  bcount 
Next  acoxmt 

Cells(103, 2)  =  lowindex  - 105 
End  Sub 


’  This  is  the  uncertainty  analysis  for  four  uncertain  variables 
'  The  worksheet  is  calculated  81  times 

Sub  Uncertainty_Anal40 
Num  =  Range("Model!$A$l").Value 
Sheets("data").Activate 
rowindex  =  105 
For  acount  =  1  To  3 
evall  =  Cells(102, 2) 

For  techn  =  1  To  Num 
Cells(4  +  evall,  techn  + 1)  =  _ 

Cells(4  +  evall  +  acount  *  25,  techn  + 1) 

Next  techn 
For  bcount  =  1  To  3 
eval2  =  Cells(102, 3) 

For  techn  =  1  To  Num 
Cells(4  +  eval2,  techn  +  1)  =  _ 

CeUs(4  +  eval2  +  bcount  *  25,  techn  + 1) 

Next  techn 
For  ccount  =  1  To  3 
eval3  =  Cells(102, 4) 

For  techn  =  1  To  Num 
Cells(4  +  eval3,  techn  +  1)  =  _ 

Cells(4  +  eval3  +  ccount  *  25,  techn  +1) 
Next  techn 
For  ecount  =  1  To  3 
eval4  =  Cells(102, 5) 

For  techn  =  1  To  Num 
Cells(4  +  eval4,  techn  +  1)  =  _ 

Cells(4  +  eval4  +  ecount  *  25,  techn  +  1) 
Next  techn 
per  =  1 

per  =  percentage(acoimt,  per) 
per  =  percentage(bcoimt,  per) 
per  =  percentage(ccount,  per) 
per  =  percentage(ecormt,  per) 

Calculate 

CeUs(rowindex,  1)  =  per 
For  techn  =  1  To  Num 
CeUs(rowindex,  techn  + 1)  =  _ 

Cells(104,  techn  +1) 
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Next  techn 

rowindex  =  rowindex  +  1 
Next  ecount 
Next  ccount 
Next  bcount 
Next  acount 

CeUs(103, 2)  =  rowindex  - 105 
End  Sub 


’  This  is  the  uncertainty  analysis  for  five  or  more 
’  uncertain  variables.  The  worksheet  is  calculated  243  times 

Sub  Uncertainty_Anal50 
Num  =  Range("Model!$A$l").  Value 
Sheets("data").Activate 
rowindex  =105 
For  acoimt  =  1  To  3 
evall  =  CeUs(102, 2) 

For  techn  =  1  To  Num 
Cells(4  +  evall,  techn  +  1)  =  _ 

Cells(4  +  evall  +  acoimt  *  25,  techn  +  1) 

Next  techn 
For  bcount  =  1  To  3 
eval2  =  Cells(102. 3) 

For  techn  =  1  To  Num 
Cells(4  +  eval2,  techn  + 1)  =  _ 

CeUs(4  +  eval2  +  bcount  *  25,  techn  +  1) 

Next  techn 
For  ccount  =  1  To  3 
eval3  =  Cells(102, 4) 

For  techn  =  1  To  Num 
Cells(4  +  eval3,  techn  +  1)  =  _ 

Cells(4  +  eval3  +  ccount  *  25,  techn  +  1) 
Next  techn 
For  ecount  =  1  To  3 
eval4  =  Cells(102,5) 

For  techn  =  1  To  Num 
Cells(4  +  eval4,  techn  +  1)  =  _ 

CeUs(4  +  eval4  +  ecount  *  25,  techn  +  1) 
Next  techn 
For  fcount  =  1  To  3 
eval5  =  Cells(102, 6) 

For  techn  =  1  To  Num 
Cells(4  +  eval5,  techn  +  1)  =  _ 

Cells(4  +  eval5  +  fcount  *  _ 

25,  techn  + 1) 
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Next  techn 
per=  1 

per  =  percentage(acount,  per) 
per  =  percentage(bcount,  per) 
per  =  percentage(ccount,  per) 
per  =  percentage(ecount,  per) 
per  =  percentage(fcount,  per) 
Calculate 

Cells(rowindex,  1)  =  per 
For  techn  =  1  To  Num 
Cells(rowindex,  techn  +  1)  =  _ 
Cells(104,  techn  + 1) 

Next  techn 

rowindex  =  rowindex  +  1 
Next  fcount 
Next  ecount 
Next  ccount 
Next  bcount 
Next  acount 

Cells(103, 2)  =  rowindex  - 105 
End  Sub 


'  This  compiles  the  data  from  the  imcertain  points  and  then 
'  combines  the  valued  that  are  within  .01  of  each  other  and 
'  adds  then  probability.  Then  the  cumulative  density  function 
'  prepared.  Finally  the  data  is  put  on  a  chart  and  displayed  to  the 
'  user.  MinBest  and  Maxbest  are  used  to  scale  the  graph  to  a  viewable 
’  chart  size. 

Sub  cdfO 

Num  =  Range(’’Model!$A$l").Value 
datapoints  =  Range("Data!$B$103").Value 
Sheets("data").Activate 
Range("N100:W199").Delete 
For  techn  =  1  To  Num 
rowindex  =  105 

Range(Cells(105, 1),  Cells(104  +  datapoints,  1  +  techn)).Select 
If  techn  =  1  Then 

Selection.Sort  Keyl:=Range("B105"),  Orderl:=  _ 
xlAscending,  Header  :=xlGuess,  OrderCustom:=l,  _ 
MatchCase:=False,  Orientation;=xlTopToBottom 
Elself  techn  =  2  Then 

Selection.Sort  Keyl:=Range("C105"),  C)rderl:=  _ 
xlAscending,  Header:=xlGuess,  OrderCustom;=l,  _ 
MatchCase;=False,  Orientation:=xlTopToBottom 
Elself  techn  =  3  Then 
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Selection.Soit  Keyl:=Range("D105"),  Or(lerl:=_ 
xlAscending,  Header:=xlGuess,  OrderCustom:=l,  _ 
MatchCase:=False,  Orientation:=xlTopToBottoin 
Elself  techn  =  4  Then 

Selection.Sort  Keyl:=Range("E105"),  Orderl:=  _ 
xlAscending,  Header;=xlGuess,  OrderCustom;=l,  _ 
MatchCase:=False,  Orientation:=xlTopToBottom 
Else 

Selection.Sort  Keyl:=Range(’F105"),  Orderl:=  _ 
xlAscending,  Header:=xlGuess,  OrderCustom:=l,  _ 
MatchCase:=False,  Orientation;=xlTopToBottom 
Endlf 

For  histindex  =  100  To  199 

If  CeUs(histindex,  13)  <  CeIls(rowindex,  1  +  techn)  Then 
Cells(histindex,  13  +  techn)  =  0 
Else 

While  Cells(histindex,  13)  >  Cells(rowindex,  _ 

1  +  techn)  And  rowindex  <  105  +  datapoints 
Cells(histindex,  13  +  techn)  =  _ 

Cells(histindex,  13  +  techn)  _ 

+  CeUs(rowmdex,  1) 
rowindex  =  rowindex  +  1 
Wend 
Endlf 

If  histindex  >  100  Then 
Cells(histindex,  13  +  techn)  =  _ 

Cells(histindex  - 1, 13  +  techn)  +  _ 

Cells(histindex,  13  +  techn) 

Endlf 

Next  histindex 
Next  techn 

ActiveSheet.ChartObjects(’'unsit’’).Delete 
minbest  =  199 
For  techn  =  1  To  Num 
rowindex  =  100 

While  CeIls(rowindex,  13  +  techn)  =  0 
techmin  =  rowindex 
rowindex  =  rowindex  +  1 
Wend 

If  techmin  <  minbest  Then 
minbest  =  techmin 
Endlf 
Next  techn 
minbest  =  minbest  -  4 
If  minbest  <  100  Then 
minbest  =  99 
Endlf 
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maxbest  =  100 
For  techn  =  1  To  Num 
rowindex  =  199 

While  CeUs(rowindex,  13  +  techn)  =  1 
techmax  =  rowindex 
rowindex  =  rowindex  - 1 
Wend 

If  techmax  >  maxbest  Then 
maxbest  =  techmax 
End  If 

Cells(minbest,  13  +  techn)  =  Cells(99, 13  +  techn) 

Next  techn 

maxbest  =  maxbest  +  5 
If  maxbest  >  199  Then 
maxbest  =  199 
End  If 

ActiveSheet.ChartObjects.Add(587, 1281, 375, 224).Select 
ActiveChart.QiartWizard  _ 

Source;=Range(CeIls(minbest,  13),  CeUs(maxbest,  _ 

13  +  Nirai)),  Gallery:=xlLine,  Format;=2,  PiotBy:=  _ 
xlColumns,  CategoryLabels:=l,  SeiiesLabels  _ 

:=1,  HasLegend:=l,  Title:="CDF",  Category'ntle:=  _ 
"Value",  ValueTitle:="Probability",  ExtraTitle:="" 
SelectioaName  =  "unsit" 

Sheets("uncerdialog").Select 
ActiveSheet.ChartObjects("unsit").Delete 
Sheets("data").Select 
ActiveSheet.ChartObjects("unsit").Select 
Selection.Copy 
Sheets("uncer^alog").Select 
ActiveSheetPaste 
Selection.Left  =  73.5 
Selection.Top  =  36.75 
Sheets("data").Select 
DialogSheets("uncerdialog").Show 
End  Sub 


Functions 


The  following  fimctions  are  called  by  the  program  and  also  by  the  various 
worksheets.  The  ValueE  function,  was  created  by  Kirkwood  and  is  found  in  Strategic 
Decision  Making  page  81. 
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’  This  formula  takes  the  five  important  characteristics 
’  for  an  exponential  value  function  and  returns  the  value 
’  This  is  for  use  with  a  monatomic  ftmction  only  (i.e.  constantly 
’  increasing  or  constantly  decreasing).  This  is  used  on  the  worksheet 
'  "Model" 

Function  ValueE(x,  Low,  High,  Monotonidty,  Rho) 

Select  Case  UCase(Monotonicity) 

Case  "INCREASING" 

Difference  =  x  -  Low 
Case  "DECREASING" 

Difference  =  High  -  x 
End  Select 

If  UCase(Rho)  =  "INFINITY"  Then 
ValueE  =  Difference  /  (High  -  Low) 

Else 

ValueE  =  (1  -  Exp(-Difference  /  Rho))  /  (1  -  Exp(-(High  -  Low)  /  Rho)) 
End  If 

End  Function 


’  This  checks  to  see  if  the  data  is  beyond  the  bounds  for  the  value 
’  of  the  exponential  value  function.  If  the  data  is  out  of  the  bounds, 
’  the  upper/  lower  bound  is  used  instead.  This  is  also  used  on  the 
’  worksheet  "Model" 

Fimction  Gheck(data,  min,  max) 

If  min  >  max  Then 
temp  =  min 
min  =  max 
max  =  temp 
End  If 

If  data  <  min  Then 
Check  =  min 
Elself  data  >  max  Then 
Check  =  max 
Else 

Check  =  data 
End  If 

End  Function 


’  This  function  carries  a  constant  multiplication  depending  on  the 
'  type  of  uncertainty  associated  with  the  point. 

'  This  is  used  in  the  subrountine  cdf 
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Function  percentage(factor,  percent) 
If  factor  =  3  Then 
percentage  =  percent  *  0.63 
Else 

percentage  =  percent  *  0.185 
End  If 

End  Function 
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